-- Logs begin at Wed 2022-08-24 14:57:45 UTC, end at Wed 2022-08-24 15:55:44 UTC. -- Aug 24 15:23:06.804484 dv-cinder-846054-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 15:23:06.816444 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 15:23:06.819441 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 15:23:06 2022] *** Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: nodename: dv-cinder-846054-2-iscsi Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: machine: x86_64 Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: clock source: unix Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: pcre jit disabled Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: detected number of CPU cores: 4 Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: current working directory: / Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: detected binary path: /usr/bin/uwsgi-core Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: your processes number limit is 47680 Aug 24 15:23:06.819596 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: your memory page size is 4096 bytes Aug 24 15:23:06.820199 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: detected max file descriptor number: 2048 Aug 24 15:23:06.820199 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: lock engine: pthread robust mutexes Aug 24 15:23:06.820199 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: thunder lock: enabled Aug 24 15:23:06.820199 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 15:23:06.820199 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 15:23:06.860279 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: Python main interpreter initialized at 0x55d2d1b5e9c0 Aug 24 15:23:06.860279 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: python threads support enabled Aug 24 15:23:06.860279 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: your server socket listen backlog is limited to 100 connections Aug 24 15:23:06.860279 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: your mercy for graceful operations on workers is 90 seconds Aug 24 15:23:06.860679 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 15:23:06.860679 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: *** Operational MODE: preforking *** Aug 24 15:23:06.861007 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: *** uWSGI is running in multiple interpreter mode *** Aug 24 15:23:06.861007 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: spawned uWSGI master process (pid: 81435) Aug 24 15:23:06.861213 dv-cinder-846054-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 15:23:06.863221 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: spawned uWSGI worker 1 (pid: 81445, cores: 1) Aug 24 15:23:06.863221 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: spawned uWSGI worker 2 (pid: 81446, cores: 1) Aug 24 15:23:06.863221 dv-cinder-846054-2-iscsi devstack@keystone.service[81435]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 15:23:11.203184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:23:11.203184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: from cryptography.utils import int_from_bytes Aug 24 15:23:11.203985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:23:11.203985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: from cryptography.utils import int_from_bytes Aug 24 15:23:11.206950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:23:11.206950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: from cryptography.utils import int_from_bytes Aug 24 15:23:11.207401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:23:11.207401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: from cryptography.utils import int_from_bytes Aug 24 15:23:11.254378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 15:23:11.254563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 15:23:11.254766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] command line args: [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 15:23:11.255030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 15:23:11.255203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ================================================================================ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 15:23:11.255453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] admin_token = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.255692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_dir = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.255874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.256053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_source = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.256232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] debug = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.256427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.256777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.256932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 15:23:11.257021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.257130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 15:23:11.257207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.257306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] command line args: [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 15:23:11.257419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.257577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 15:23:11.257655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.257760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ================================================================================ {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 15:23:11.257850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_config_append = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.257966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] admin_token = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_dir = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] config_dir = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] config_source = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] debug = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.258941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.259874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_config_append = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_dir = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_file = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.260961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.261993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] publish_errors = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.262935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] standard_threads = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.263899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_journal = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_json = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.264883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_stderr = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] publish_errors = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_syslog = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.265633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.265883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.265965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.266121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.266201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.266358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] standard_threads = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.266436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.266591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.266692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.266866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.266948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.267046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.267125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.267215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] use_journal = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.267356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.267442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] use_json = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.267546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] use_stderr = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.267624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.267723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] use_syslog = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.267837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:23:11.268070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.268947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.269923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.270996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.271912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.272996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.273967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.274991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.275907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.276998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.277995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.278954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.279957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.280923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.281894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.282991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.283943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.284965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.285907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.286865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.287939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.288951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.289900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.external = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.290860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.password = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.token = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.291960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.292992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.external = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.293963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.password = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] auth.token = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.294936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.295134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.295221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.295340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.295926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.296960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.297916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.298889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.299852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.300972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.301983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.302874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.303877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.304987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.305922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.306975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.307938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.308995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.309940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.310934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.311086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.311239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.311373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.312810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.313957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.314964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.315915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.316930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.317885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.318992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.319987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.320864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.321966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.322963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.323992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.324933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.325892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.326936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.327985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] role.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] role.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.328906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.329906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.330889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.331970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.332952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.333968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.334934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.335893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.336988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.337947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.338942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.339948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.340890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.connection = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.341983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.342973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 15:23:11.343053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.343243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.343806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.344050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.344298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.344537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.344788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.345024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.345273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.345507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.345748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.345983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:23:11.346152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81446) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 15:23:11.372774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.373030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.373221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.375465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.375699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.375924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.381928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.384368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.411793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.412020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.412881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.413113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.466717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.467865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.516933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.517198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.518134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.518372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.674239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.674513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.674722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.674899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.675951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.676167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.698748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.698981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.699871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.700667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.711616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.711901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.725109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.725281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.725497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.725715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.725975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.726925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.727057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.727276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81446) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:23:11.742907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.743115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:23:11.743458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.743656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:23:11.743864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:23:11.744297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:23:11.744733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.744991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.745119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.745615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.745881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.754925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.755204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.755473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.755751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.755882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.756149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.756424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.756711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.762654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.764341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.765809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.766092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.766374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.766673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.766962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.767223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.767361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.767494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.767627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.767757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.767898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.768905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.769151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.769538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.769815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.770087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.791593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.793049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.793295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.793428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.794784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.795071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.798043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.798321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.798602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.799921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.800209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.800496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.804921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.805206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.806970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.807262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.809282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.809572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.809854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.810136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.810410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.810705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.810975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.811266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.811465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.811761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.812045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.812332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.812614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.812893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.813167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.813437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.828847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.829122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.831433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.831716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.832998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.835595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.836231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.836777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.839087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.839387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.840972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.843915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.844183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.847058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.847420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.847723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.848006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.848286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.848569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.848853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.850297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.850606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.850919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.851214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.851503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.851794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.865550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.865849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.866114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.866375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.868311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.868602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.868871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.869136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.873219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.875491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.875770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.876745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.878292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.878374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.878573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.878679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.878946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.879235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.879524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.881237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.881523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.881817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.882111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.882403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.890851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.891132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.891406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.891681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.891957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.892227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.892499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.893985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.894270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.894552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.894858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.895144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.895442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.895728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.907806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.910847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.911337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.913732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.914365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.915200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.916671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.916953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.917247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.917330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.918720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.920291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.921489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.921856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.924244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.925706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.928359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.928649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.928939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.929224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.929500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.929813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.932711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.933010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.933299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.933586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.938253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.938542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.938850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.942211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.942506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.942815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.945689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.945975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.946255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.946526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.946831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.947110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.947378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.947642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.947918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.948198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.948464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.948721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.948998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.949254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.949708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81446) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:23:11.949994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.950278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.950557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.950896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.951219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.951496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.951878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.952255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.952552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.952831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.953108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.953377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.953642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81446) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:23:11.983979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2d1b5e9c0 pid: 81445 (default app) Aug 24 15:23:11.987347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d365e-1bd6-4af9-9462-019d25018381 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:11.987632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d365e-1bd6-4af9-9462-019d25018381 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:11.987905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d365e-1bd6-4af9-9462-019d25018381 None None] PATH_INFO: `/v3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:11.988940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d2d1b5e9c0 pid: 81446 (default app) Aug 24 15:23:11.992095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Wed Aug 24 15:23:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:18.555036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:18.555552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:18.555638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:18.559852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:23:18.568506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:18.568959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:18.569044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:18.641667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: INFO dbcounter [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] Registered counter for database keystone Aug 24 15:23:18.660857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG oslo_db.sqlalchemy.engines [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 15:23:18.667121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writer thread running {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 15:23:18.849229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG passlib.handlers.bcrypt [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81445) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 15:23:18.849639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG passlib.handlers.bcrypt [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81445) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 15:23:18.905543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:18.907282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:19.035261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 15:23:19.488346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:19.557825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: INFO dbcounter [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] Registered counter for database keystone Aug 24 15:23:19.581281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG oslo_db.sqlalchemy.engines [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81446) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 15:23:19.587720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writer thread running {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 15:23:19.707012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d241-cfb9-409e-a9c3-fa983f0cbdc5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:19.711209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:19.711816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:19.712099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:19.712623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:19.721202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:19.773267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING keystone.common.rbac_enforcer.enforcer [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 15:23:19.773606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be8325-95a8-416b-9ad9-167c175dde1e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:19.783744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 15:23:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:23:19.789784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:19.805679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e0d31-cb22-4212-ad66-54f30d2101f6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:19.809981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:19.811115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:19.811400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:19.811676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] PATH_INFO: `/v3/projects/5e88980106c84e4fadc67759de8e9449` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:19.813472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] RBAC: Authorizing `identity:get_project(project_id=5e88980106c84e4fadc67759de8e9449)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:19.867140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING keystone.common.rbac_enforcer.enforcer [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 15:23:19.867421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c55d3-6de9-4866-95ae-80eee117d0bd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:19.871999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Wed Aug 24 15:23:19 2022] GET /identity/v3/projects/5e88980106c84e4fadc67759de8e9449 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:21.776080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc320-edc5-4472-9c69-b08941c8b6cc None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:21.776632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc320-edc5-4472-9c69-b08941c8b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:21.776632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc320-edc5-4472-9c69-b08941c8b6cc None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:21.781351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:21.786910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:21.787316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:21.787482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:21.807153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:21.808675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:21.816780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 15:23:22.087356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebc320-edc5-4472-9c69-b08941c8b6cc None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:22.100112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebc320-edc5-4472-9c69-b08941c8b6cc None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:22.104315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:22.104915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:22.105198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:22.105472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:22.113670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:22.115486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e80d-8b4a-46a5-a659-6cb7affee423 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:22.124163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 15:23:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:23:22.129603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:22.138530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e7415-bead-4c43-9362-1ffe5b30c705 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:22.142836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:22.143453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:22.143733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:22.144004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:22.145305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:22.147154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092ce16-5b0d-4e88-8b1e-17152f623e55 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:22.148666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Wed Aug 24 15:23:22 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:24.116794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f41cf-5f32-475a-8dfb-fa44d736e3bf None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.117324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f41cf-5f32-475a-8dfb-fa44d736e3bf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.117324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f41cf-5f32-475a-8dfb-fa44d736e3bf None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.118670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:24.123732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.124133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.124313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.142468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:24.143655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:24.175254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd72d-56ab-410d-9839-ee32dfb4f98d None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.175547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd72d-56ab-410d-9839-ee32dfb4f98d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.175827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd72d-56ab-410d-9839-ee32dfb4f98d None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.177324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:23:24.182236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a139-b682-41af-b13e-6464234326e0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.182591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a139-b682-41af-b13e-6464234326e0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.182958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a139-b682-41af-b13e-6464234326e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.224907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:24.226074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG passlib.handlers.bcrypt [None req-a572a139-b682-41af-b13e-6464234326e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81446) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 15:23:24.226482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG passlib.handlers.bcrypt [None req-a572a139-b682-41af-b13e-6464234326e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81446) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 15:23:24.272544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a572a139-b682-41af-b13e-6464234326e0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:24.274133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a572a139-b682-41af-b13e-6464234326e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:24.374064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:24.493427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.518140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:24.602113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3bdca-ecae-4a10-ad93-6916e5173b81 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:24.634064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:24.634626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.634932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.635208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.646813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:24.650523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572a139-b682-41af-b13e-6464234326e0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:24.655550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:24.656664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:24.657536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:24.662276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.741677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:24.754422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572a139-b682-41af-b13e-6464234326e0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:24.756756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:24.760625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13dc3d-0c60-4184-a22b-df6a3affbf2c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:24.793799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:24.794671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.794788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:24.794970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.795249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.795374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.795651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.795923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.797079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:24.797605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:24.799120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:24.811027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:24.817392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.864477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:24.865979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:24.917068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48205fe3-6be7-4286-b89e-ba2294cdb11f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:24.951877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:24.952512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:24.952813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:24.953725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:24.955030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:24.956492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:24.958073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 9/16] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:24.963981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.964647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 8/17] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:24.970112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:24.972881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:24.978900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:25.063656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cc783-0cad-48be-b28a-50e00b18540d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:25.067372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7aee-7b35-4d1f-bb37-57d061a6759f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:25.098137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:25.098803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:25.099088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:25.099361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:25.100515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:25.101181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:25.101985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:25.102072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:25.102312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:25.102589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:25.103558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 10/18] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:25.109478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:25.121082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:25.122552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:23:25.141966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:25.142917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:23:25.143737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:23:25.143737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:23:25.143737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:25.143737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 9/19] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:23:25.148752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:25.199991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:25.206246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:25.220361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7620010-2131-4e78-898b-7a2a196592b7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:25.250383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55c968-cbb2-4ce7-b4da-cda7b2c73a4e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:25.254836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:25.255440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:25.255723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:25.255996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:25.256758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.domain.id=default, user.id=75821779c93e4039bcee682de1f82886) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:25.257073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:25.258541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e1353-15fe-49fa-bef1-523bdcc45c43 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:25.265329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 15:23:25 2022] GET /identity/v3/role_assignments?user.id=75821779c93e4039bcee682de1f82886&scope.domain.id=default&role.id=826e2155943c48ab9c01f294b1734d83 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:25.284612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:25.285250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:25.285529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:25.285800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:25.286464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:25.286807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:25.288235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935894f-44ed-4e9c-a8cc-daf1302b8a0e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:25.310968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:27.925374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80769dd-d39c-4cdd-9af9-819c92084188 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:27.925902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80769dd-d39c-4cdd-9af9-819c92084188 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:27.925902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80769dd-d39c-4cdd-9af9-819c92084188 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:27.927577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:27.932483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0a0-2667-45f5-b639-bd7a485324ae None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:27.932885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0a0-2667-45f5-b639-bd7a485324ae None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:27.933068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d0a0-2667-45f5-b639-bd7a485324ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:27.959656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6120d0a0-2667-45f5-b639-bd7a485324ae None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:27.960865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6120d0a0-2667-45f5-b639-bd7a485324ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:28.011000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcee6-1857-4bd1-875c-047a61d201d1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.011305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcee6-1857-4bd1-875c-047a61d201d1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.011576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebcee6-1857-4bd1-875c-047a61d201d1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.013178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 13/23] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:28.019018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9739-3539-4859-8b4a-ddc99e304eab None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.019314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9739-3539-4859-8b4a-ddc99e304eab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.019598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb9739-3539-4859-8b4a-ddc99e304eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.054103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4ecb9739-3539-4859-8b4a-ddc99e304eab None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:28.055347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4ecb9739-3539-4859-8b4a-ddc99e304eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:28.069423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 11/24] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:28.070897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648cb2d-a269-45c0-b58a-0e13c1a0d9ac None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.071183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648cb2d-a269-45c0-b58a-0e13c1a0d9ac None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.071456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648cb2d-a269-45c0-b58a-0e13c1a0d9ac None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.076818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 12/25] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:28.081520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f19e2-5fa4-47c2-aad0-9e9c2220acb4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.081799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f19e2-5fa4-47c2-aad0-9e9c2220acb4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.082068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f19e2-5fa4-47c2-aad0-9e9c2220acb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.115946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-048f19e2-5fa4-47c2-aad0-9e9c2220acb4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:28.117170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-048f19e2-5fa4-47c2-aad0-9e9c2220acb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:28.160583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 14/26] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:28.209076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:28.243872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 13/27] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:28.403682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389be21-6cae-4060-beaa-fc91d69f610d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.404012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389be21-6cae-4060-beaa-fc91d69f610d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.404238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389be21-6cae-4060-beaa-fc91d69f610d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.405793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 15/28] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:28.411780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.412330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.412804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.443637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:28.445186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:28.520277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389be21-6cae-4060-beaa-fc91d69f610d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:28.533148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 14/29] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:28.605562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:28.668280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:28.733819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389be21-6cae-4060-beaa-fc91d69f610d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:28.804049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:28.804593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.804879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.805156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.805842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:28.807274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:28.821188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce28ca-afb8-44c4-b207-ec7083573e5e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:28.830916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:28.876748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:28.877381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:28.877661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:28.877935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:28.889476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:28.896339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:28.897244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:28.897990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 15/30] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:23:28.898617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:28.960597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 16/31] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 15:23:28 2022] POST /identity/v3/roles => generated 225 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:28.963692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.060518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856eec3-b0fe-47c0-8558-eeef36fd58f5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.101110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.101754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.102036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.102358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.114460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.115986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:23:29.132345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:29.133354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:23:29.134803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:23:29.134803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:23:29.134803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:29.134803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 16/32] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:29.134803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.186102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.213048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56108d19-7a12-40f5-b9f1-69162dada973 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.253711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.254333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.254620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.254949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.255636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:29.255951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.257569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.258945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.268195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 17/33] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:29.269228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.280997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9499fb-c29d-4bfa-848d-e37a6e9da399 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.319537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.320341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.320624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.320898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.321572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.322988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207d650-64f4-4a7b-8ee9-05bfe2ffef0c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.369885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dded71-7448-4989-ad49-827bf71d10e9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.404443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.405050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.405331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.405603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.406274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:29.407398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.408765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d205afc-ad59-43fe-90e8-d0402056ce41 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.425496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 18/34] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:29.426757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738911-6b65-4dfe-9d34-e9832a6f1a12 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.427047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738911-6b65-4dfe-9d34-e9832a6f1a12 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.427324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738911-6b65-4dfe-9d34-e9832a6f1a12 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.428708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 19/35] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:29.429620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9890-0715-496c-9c39-c783012c111e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.429964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9890-0715-496c-9c39-c783012c111e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.430250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc9890-0715-496c-9c39-c783012c111e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.431479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 20/36] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:29.434769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc9890-0715-496c-9c39-c783012c111e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.446391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 17/37] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 15:23:29 2022] POST /identity/v3/roles => generated 219 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:29.447862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.448159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.448439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.483237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:29.484986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:29.513875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.563721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 18/38] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:29.564906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.582992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc9890-0715-496c-9c39-c783012c111e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.584835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.632568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.633219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.633502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.633781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.635074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.636546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea927575-1bc0-49a2-abdf-b47c4407673e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.638435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 21/39] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:29 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:29.639419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.639736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.640014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.663012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:29.664835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:29.698448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38114d8f-cd54-4011-a894-89685d9f6f90 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.739487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.740106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.740395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.740673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.741560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.743281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.750932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 22/40] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:29.750932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.786824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 19/41] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:23:29 2022] POST /identity/v3/projects => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:29.843178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.857267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.898487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dfb5d-f2ec-4679-bdb1-3750f3711ac5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:29.933076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:29.950352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:29.951087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:29.951367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:29.951639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:29.954315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:29.957093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:29.959134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 23/42] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:29.965152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:29.997080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a100649-e85b-462e-9273-6545620b7c2c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:30.046359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:30.047628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:30.047915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:30.048194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:30.048874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:30.050284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:30.065473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 20/43] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 15:23:29 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:23:30.066300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:30.089823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf945-12a5-4ebf-a7dd-c83d244b8b2d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:30.118814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:30.134096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:30.135160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:30.135723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:30.136014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] PATH_INFO: `/v3/domains/default/users/75821779c93e4039bcee682de1f82886/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:30.138542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=75821779c93e4039bcee682de1f82886, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:30.140275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:30.156171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 24/44] 10.222.0.73 () {58 vars in 1361 bytes} [Wed Aug 24 15:23:29 2022] PUT /identity/v3/domains/default/users/75821779c93e4039bcee682de1f82886/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:30.186405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3e43d-611c-45a4-a2ad-dc0fe2713843 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:30.212143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:30.236057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:30.236724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:30.237011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:30.237289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:30.238533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:30.240111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52b66f-5bdb-49f0-a08b-7aca01a061d4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:30.241782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 21/45] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:30.247578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:30.363485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816dab4-6f4e-4aae-a01c-50408b0ec2f5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:30.396141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:30.396672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:30.396954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:30.397232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:30.397883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:30.399382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928cc52-76c0-40d6-ad08-12f850a8c71b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:30.429577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 25/46] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:23:30 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:32.078069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efab57-f543-4134-9310-0dea056a33fb None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.079114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efab57-f543-4134-9310-0dea056a33fb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.079195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efab57-f543-4134-9310-0dea056a33fb None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.080493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 22/47] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:32.085719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107de8f-7373-4cd0-8006-61447f09f93f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.086132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107de8f-7373-4cd0-8006-61447f09f93f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.086398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107de8f-7373-4cd0-8006-61447f09f93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.210143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8107de8f-7373-4cd0-8006-61447f09f93f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:32.211391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8107de8f-7373-4cd0-8006-61447f09f93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:32.239253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:32.280089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 26/48] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:32.360505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f5c94-4b8c-4828-813e-c68e0b19f3b8 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.360716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f5c94-4b8c-4828-813e-c68e0b19f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.360997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f5c94-4b8c-4828-813e-c68e0b19f3b8 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.362345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 23/49] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:32.367632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.367925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.368204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.386607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:32.387806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:32.453912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 27/50] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:32.550419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f5c94-4b8c-4828-813e-c68e0b19f3b8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:32.666205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f5c94-4b8c-4828-813e-c68e0b19f3b8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:32.699309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:32.699940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.700224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.700500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.711476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:32.716524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:32.717494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:32.718418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 24/51] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:32.725257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:32.735218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:32.784182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:32.818902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:32.826024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ba003-40b7-4b16-a46d-0e43bd37e55f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:32.836958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200a9-e902-4791-8047-551269c8439d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:32.860684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:32.861288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.861570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.861838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.862498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:32.862837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:32.864212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:32.871323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:32.871967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:32.872079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 28/52] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:32.872268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:32.872506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:32.873623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:32.875258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:32.878684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 25/53] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:32.878822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:32.882368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:32.982346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5048e28-0fba-41e2-8a23-58f987f033a8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:32.988561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac037-4dd7-45fb-ac56-f50fdb3bfeef None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:33.004338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:33.015854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:33.016987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:33.017279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:33.017557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:33.018742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:33.020202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:33.021799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 29/54] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:32 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:33.022888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:33.023500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:33.023786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:33.024068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:33.024718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:33.026170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:33.027448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:33.054202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:33.134057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac982c-8aa1-490b-bbae-b1cbaf43ba07 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:33.160374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 26/55] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 15:23:32 2022] POST /identity/v3/projects => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:23:33.171709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:33.172551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:33.172838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:33.173168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:33.174347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:33.175904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9261f-e0ce-409b-9f56-43f9bd52a995 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:33.177608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 30/56] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:33.183908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:33.288536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1993-3ea4-4e5b-a093-ffa4e1678482 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:33.328201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:33.329406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:33.329689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:33.329959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:33.330919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.domain.id=default, user.id=75821779c93e4039bcee682de1f82886) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:33.331226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:33.332686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50f3f-b898-498e-9e09-cfa1a7b31fca None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:33.342913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 27/57] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 15:23:33 2022] GET /identity/v3/role_assignments?user.id=75821779c93e4039bcee682de1f82886&scope.domain.id=default&role.id=826e2155943c48ab9c01f294b1734d83 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:35.085844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac1f-731c-48b1-9149-b735b349f073 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:35.086363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac1f-731c-48b1-9149-b735b349f073 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:35.086490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ac1f-731c-48b1-9149-b735b349f073 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:35.087878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 31/58] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:35.093391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:35.093788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:35.093964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:35.112873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:35.114079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:35.139944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:35.180937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 28/59] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:35.448126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ac1f-731c-48b1-9149-b735b349f073 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:35.540591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:35.550438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ac1f-731c-48b1-9149-b735b349f073 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:35.584397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:35.585035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:35.585322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:35.585599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:35.586751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:35.588205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c0dc4-c011-4ba1-a9f9-d2824a58735b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:35.590035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 32/60] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:35.595642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:35.692633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8319f-cf45-4768-87a7-91714f7b9cd5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:35.725596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:35.726141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:35.726421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:35.726726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:35.727380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:35.728818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3ac11-41a4-4b02-a44d-17fa10e80b4d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:35.759443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 29/61] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:23:35 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:23:38.715344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6064-cbe4-42f1-94e9-d5ad4474e13d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:38.716333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6064-cbe4-42f1-94e9-d5ad4474e13d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:38.716817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6064-cbe4-42f1-94e9-d5ad4474e13d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:38.718386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 33/62] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:38.724882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d29-f9a8-4c31-a0b2-6f1ddd54f497 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:38.725742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d29-f9a8-4c31-a0b2-6f1ddd54f497 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:38.726241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0d29-f9a8-4c31-a0b2-6f1ddd54f497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:38.751485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-43cc0d29-f9a8-4c31-a0b2-6f1ddd54f497 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:38.753001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-43cc0d29-f9a8-4c31-a0b2-6f1ddd54f497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:38.771126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:38.847313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 30/63] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:38.870424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c92e5-2227-4fe1-a4ef-9a0b1a6c60e0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:38.871070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c92e5-2227-4fe1-a4ef-9a0b1a6c60e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:38.871610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c92e5-2227-4fe1-a4ef-9a0b1a6c60e0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:38.873418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 34/64] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:38.879218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db23069-2c56-47d2-b423-532c192d2350 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:38.879764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db23069-2c56-47d2-b423-532c192d2350 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:38.880222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db23069-2c56-47d2-b423-532c192d2350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:38.906940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1db23069-2c56-47d2-b423-532c192d2350 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:38.908710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1db23069-2c56-47d2-b423-532c192d2350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:39.004090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 31/65] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:39.179944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c92e5-2227-4fe1-a4ef-9a0b1a6c60e0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.261304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3971a8-859f-4dd6-9383-464102e33ff4 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.261592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3971a8-859f-4dd6-9383-464102e33ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.261787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3971a8-859f-4dd6-9383-464102e33ff4 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.263170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 32/66] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:39.276749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d1bc-dedf-43d1-a674-d36ff44c9e41 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.277336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d1bc-dedf-43d1-a674-d36ff44c9e41 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.277614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d1bc-dedf-43d1-a674-d36ff44c9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.296843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-24e7d1bc-dedf-43d1-a674-d36ff44c9e41 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:39.298456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-24e7d1bc-dedf-43d1-a674-d36ff44c9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:39.303279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c92e5-2227-4fe1-a4ef-9a0b1a6c60e0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.342740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.343519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.344020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.344521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.357806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a1737c84-61b5-4719-a253-85632733d216 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:39.363238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:39.364184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:39.364963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 35/67] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:39.365589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae314-fb13-489a-bdf3-fd55bf12e0ee None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.365884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae314-fb13-489a-bdf3-fd55bf12e0ee None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.366312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaae314-fb13-489a-bdf3-fd55bf12e0ee None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.368029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 36/68] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:39.373072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaae314-fb13-489a-bdf3-fd55bf12e0ee None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.388805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 33/69] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:39.395090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.395497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.395876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.400531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.415242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:39.416428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:39.424271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.487491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 34/70] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:39.488484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaae314-fb13-489a-bdf3-fd55bf12e0ee None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.488776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.525352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.526291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.526856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.527330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.528237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:39.528736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:39.530306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:39.541393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 37/71] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:39.547250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.593900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d12e72-8ba3-4c87-91eb-33f639fce0f2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.630218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.631316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.631764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.632208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.639685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.646329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b61ae85a-730c-4bb8-806b-ca80ebdd4d5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:39.653236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:39.654112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:39.654917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:39.657418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 35/72] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:39.659806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87ad7-1299-4e2a-9885-bc2212ab26ec None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.660267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87ad7-1299-4e2a-9885-bc2212ab26ec None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.660697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87ad7-1299-4e2a-9885-bc2212ab26ec None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.662461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 36/73] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:39.664298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.665171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87ad7-1299-4e2a-9885-bc2212ab26ec None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.694162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d6c75-72d9-43c9-8325-b49f04f56fd8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.733097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.733855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.734034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.734300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.750028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:39.751589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c94732-6a12-428d-b9b5-a9d712d68340 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:39.755017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 38/74] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:39.755017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.755017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.755017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.772898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:39.774055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:39.779866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87ad7-1299-4e2a-9885-bc2212ab26ec None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.816847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.817460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.817744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.818022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.818724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:39.819337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:39.820722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:39.832836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 37/75] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:39.833633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.845954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 39/76] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:39.846738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:39.910146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.917986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:39.951923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159e2f3-c6f2-46e2-9e8d-92f380a2607b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.955803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bc7c2-7dc2-4c68-ac47-03b732d20029 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:39.989582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.990216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.990607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.990910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:39.991114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:39.991802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:39.992026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:39.992306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.003534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:40.008456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:40.009484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:40.010269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 38/77] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:40.010948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.079842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.081500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.128583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055143-8d90-4dbc-80b7-47e74f9376de None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.149014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.153543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.164366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.164997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.165283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.165558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.177053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:40.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:40.182728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:40.183494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:40.185929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 40/78] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:40.186791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.187857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 39/79] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:40.188819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.285230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0002be-a45f-465b-8dd7-c240b7d11629 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.286458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77745444-8f78-4cc5-a788-25775e22b33b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.323679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.324321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.324607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.324888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.326016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.326612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.327255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.327505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.327599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.327796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.328523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:40.328832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.329098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 41/80] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.329934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.330440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.345141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 40/81] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.345141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.374955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.383447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.437355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e99cf-dbd7-4ca7-b06c-0d64aeae1123 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.440865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a678e76-ac03-466a-89f3-145eda405530 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.472518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.473153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.473436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.473714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.474859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.475177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.475780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.476063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.476303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.476391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.476992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:40.477300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.477870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 42/82] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.478627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.478757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.489786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 41/83] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.490587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.578729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e028-dce1-4603-af0a-b5b3149a3fb5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.586089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.590967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac7fe-5d32-4b11-8b18-86be21c51777 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.593293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.615307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.615953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.616241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.616516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.617659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.619154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.620994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 43/84] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.621563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.631959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.632577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.632864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.633147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.707998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.709502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.723198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beee918-5b14-46f5-9905-971d3cbf6083 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.758390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.759046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.759328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.759608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.760900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.762358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.764012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 44/85] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.764766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.813475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.814965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 42/86] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:40.819952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.819952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:40.877181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcfc10-0648-4105-9be0-2b5f84f381ae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.911713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.912315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.912600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.912874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.913301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629a1e8-5ae1-4b66-9fee-65709214ac1a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:40.913516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.914993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.942882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 45/87] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:23:40 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:40.943705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:40.952015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:40.952678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:40.952963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:40.953240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:40.955498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:40.957020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:40.958983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 43/88] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:40.959716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.036370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.065170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.070795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596f1e5-f057-43f8-afd1-685a9bee770c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.070983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4a1b2-cae8-4dfa-9a33-e62d3f48e764 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.113775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.114393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.114775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.115056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.122084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.122759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.123052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.123335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.124555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:41.124873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.126417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.137342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 46/89] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:41.138078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.206075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.207626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.284817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c221d4-27b4-426d-bdd9-64ec5f368e88 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.309121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.334897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.335544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.335825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.336096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.337568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.338743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.340925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 47/90] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:41.341976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.355136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 44/91] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:41.356677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.382045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.478143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3e8ef-9622-4a8b-85ba-b21d702e0b7f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.479196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec8e41-b54a-4d06-a71d-f3b11c72976f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.520561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.521659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.522112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.522527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.545099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.545732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.546016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.546295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.547121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=75821779c93e4039bcee682de1f82886) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:41.547441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.550079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.562322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 45/92] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/role_assignments?user.id=75821779c93e4039bcee682de1f82886&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:41.563175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.608600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.611032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.640797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.660317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:41.784565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 48/93] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:41.788396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cf6b9-664b-47ab-946f-4911ec01ee34 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.790991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.848561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:41.849207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:41.849481 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:41.849761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] PATH_INFO: `/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:41.850957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=a1b5d6c51cda4ac682c8f2e91acd7d8c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:41.854375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:41.854375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 46/94] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c => generated 312 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:41.866675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:41.976032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ddcc6-e0fe-44a4-9b53-a11475244ac1 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:41.980056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:42.012928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:42.013572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:42.013859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:42.014140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:42.015022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=a1b5d6c51cda4ac682c8f2e91acd7d8c, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:42.015339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:42.017426 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:42.023301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 47/95] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=a1b5d6c51cda4ac682c8f2e91acd7d8c&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:42.023779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a2329-a641-4e53-b732-4481691fee32 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:42.082614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:42.090751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:42.091761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:42.092243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:42.092749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:42.094191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:42.095977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24297a4-c4b5-4630-8a24-cfbdb636b4fa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:42.097936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 49/96] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:41 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:42.103826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:42.223290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ecf9f-b6c6-4ae2-9465-b5500d463f87 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:42.258866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:42.259475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:42.259764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:42.260043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:42.260780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:42.261095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:42.262534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1c542-461f-4a42-80c6-1257a9674850 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:42.268475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 48/97] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:42 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:43.698286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c395d-0b4a-4c76-9aba-e8dc5c2210e4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:43.698781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c395d-0b4a-4c76-9aba-e8dc5c2210e4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:43.698781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c395d-0b4a-4c76-9aba-e8dc5c2210e4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:43.700127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 50/98] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:43.705375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98a8be-b76c-40ee-b1c6-8d32cfaca61c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:43.705770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98a8be-b76c-40ee-b1c6-8d32cfaca61c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:43.706047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98a8be-b76c-40ee-b1c6-8d32cfaca61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:43.725177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0e98a8be-b76c-40ee-b1c6-8d32cfaca61c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:43.726449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0e98a8be-b76c-40ee-b1c6-8d32cfaca61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:43.764979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:43.816851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 49/99] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:44.092962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bcef-1cc1-45f6-ac6f-b9ce50386ae7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.093323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bcef-1cc1-45f6-ac6f-b9ce50386ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.093517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7bcef-1cc1-45f6-ac6f-b9ce50386ae7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.094902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 51/100] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:44.112787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737fe3-541e-437a-bf6c-c5af49e1da9c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.113074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737fe3-541e-437a-bf6c-c5af49e1da9c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.113355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737fe3-541e-437a-bf6c-c5af49e1da9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.135000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ad737fe3-541e-437a-bf6c-c5af49e1da9c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:44.136177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ad737fe3-541e-437a-bf6c-c5af49e1da9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:44.228691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 50/101] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:44.275733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7bcef-1cc1-45f6-ac6f-b9ce50386ae7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:44.419407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7bcef-1cc1-45f6-ac6f-b9ce50386ae7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:44.453975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f87bf-77c4-4130-ba7c-ef7388333c84 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.454319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f87bf-77c4-4130-ba7c-ef7388333c84 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.454543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f87bf-77c4-4130-ba7c-ef7388333c84 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.456250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 51/102] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:44.462502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.462824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.463113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.474068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:44.475010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.475491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.475956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.495599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:44.501190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:44.502214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:44.503025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:44.506713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:44.506713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:44.508548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 52/103] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:44.514703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:44.632369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 52/104] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 15:23:44.633231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:44.678071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:44.687001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:44.769811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f66e6-4170-4bfe-ae8c-849b974eac86 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:44.807712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a22144-8b62-4c15-b443-9814d38125b2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:44.813762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:44.815401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.815694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.815975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.821822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:44.823341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd1ddc-2e91-477f-bb9f-0b0881baa0b3 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:44.825124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 53/105] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:44.828096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb276c7-afe7-4a1f-b5c9-b9aa2a4ddb30 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.828383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb276c7-afe7-4a1f-b5c9-b9aa2a4ddb30 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.828661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb276c7-afe7-4a1f-b5c9-b9aa2a4ddb30 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.831958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 54/106] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:44.833573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb276c7-afe7-4a1f-b5c9-b9aa2a4ddb30 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:44.869068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:44.869954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.870239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.870519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.871400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:44.871709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:44.873676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e124705-8a0d-4910-a823-ce470dfdd15a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:44.884112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 53/107] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:44.887874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290f11-a430-4e82-b6f8-6d67ae19792c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.888236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290f11-a430-4e82-b6f8-6d67ae19792c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.888589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290f11-a430-4e82-b6f8-6d67ae19792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:44.911925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f3290f11-a430-4e82-b6f8-6d67ae19792c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:44.913291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f3290f11-a430-4e82-b6f8-6d67ae19792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:44.956984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:44.970556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb276c7-afe7-4a1f-b5c9-b9aa2a4ddb30 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:44.992508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 54/108] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:44.996352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e88a-96c4-4252-bff4-69ed2c1f2a87 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:44.997119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e88a-96c4-4252-bff4-69ed2c1f2a87 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:44.998043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e88a-96c4-4252-bff4-69ed2c1f2a87 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.000153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 55/109] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:45.002341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e88a-96c4-4252-bff4-69ed2c1f2a87 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.010413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.011295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.011646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.011997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.012859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.014577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.046195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 55/110] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:23:44 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:45.047793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.071890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.140059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e88a-96c4-4252-bff4-69ed2c1f2a87 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.183246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.184229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.184733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.185191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.186742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.188505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd71e53-e551-44ae-b220-72af076a96f0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.191086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 56/111] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:45.192556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.193019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.193461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.200733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342798e2-0071-4e81-8831-a5483b9b10ca None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.216013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:45.217178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:45.245460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.246102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.246390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.246685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.262173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-03d74a92-999f-43fd-8b02-203fe2e72132 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:45.267953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:45.268908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:45.270088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 56/112] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:45.271016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f545dc-338c-4b4d-907e-60a4c361cbba None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.271016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f545dc-338c-4b4d-907e-60a4c361cbba None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.271133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f545dc-338c-4b4d-907e-60a4c361cbba None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.272481 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 57/113] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:45.273012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f545dc-338c-4b4d-907e-60a4c361cbba None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.279761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.317941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 57/114] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:45.317941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.381019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.411454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f545dc-338c-4b4d-907e-60a4c361cbba None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.444970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b0ba8-950f-4cae-830b-5320a8c82ac6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.456835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.458262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.458821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.459231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.460784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.463165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08284da3-5ef8-4064-bf3c-4215c2f125e3 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.465870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 58/115] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:45.467241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.467663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.468095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.485159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.485877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.486356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.486899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.487619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:45.488019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.489649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.491660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:45.493416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:45.499726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 58/116] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:45.500639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.577701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.590107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 59/117] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:45.598594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.631825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb2d39-e77c-41d6-a4aa-1232183b2f08 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.647083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.684000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.685407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.685924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.686341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.703518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:45.712033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:45.713067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:45.713991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:45.717210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 59/118] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:45.718166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.742183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c9264-fefc-478a-b522-09336fdc81c0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.792032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.792782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.792977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.793254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/1a7f29f505094c969219499c619c7462/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.795933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, user_id=1a7f29f505094c969219499c619c7462, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.797713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.818859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 60/119] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:45 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/1a7f29f505094c969219499c619c7462/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:45.819982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.865336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8252-34bf-4fa5-9174-f5c94a3266c5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.918400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:45.919119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:45.919403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:45.919743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] PATH_INFO: `/v3/users/886c0bbbf0784048a18fb7be815dc421` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:45.927238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:45.938330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] RBAC: Authorizing `identity:get_user(user_id=886c0bbbf0784048a18fb7be815dc421)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:45.941691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:45.943896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 60/120] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/users/886c0bbbf0784048a18fb7be815dc421 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:45.944906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:45.960185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab21a7-1669-43bf-9d05-48cd394dfcfd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:45.983532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.012729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.014371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.014828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.015267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.028029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:46.034271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:46.035249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:46.036011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 61/121] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:46.036632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.086689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ddb5-ab50-4a27-835e-a58df3de4c50 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.125228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.126087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.126379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.127195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.127890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:46.128207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.129808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.139408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 61/122] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.144406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.144661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1ccbc-501c-4184-8eec-908aead0d39a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.166266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.181852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.182676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.182792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.183052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.195008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:46.200067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:46.201028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 62/123] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:46.201941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.259022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.265248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f250c-05cc-4c45-9501-fef972736754 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.305059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.306707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.307198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.307650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.308336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61e69-988c-42b6-b2f4-4cbcb62d2356 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.352855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.353466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.353747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.354019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.354730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:46.355037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.356477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.372639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 63/124] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.372639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.414468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.416731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.417560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.492005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed886314-adf5-4aff-b1b6-6b2278e90238 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.529288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.533899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.534722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.535012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.535302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.536707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.538919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.545046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 64/125] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.545046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.547682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 62/126] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:46.548641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.667284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8613b-cbfe-439e-8472-bbbd04b647b2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.676468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cf25-e64e-4e50-a0fc-8dfd6c2b0e2c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.687095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.708209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.708837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.709125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.709404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.710082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:46.710533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.712615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.721306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.722257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.722787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.723261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.724702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.726357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.727412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 65/127] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.728267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.729284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 63/128] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.730349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.813864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:46.865860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19b78-1a36-4c94-9c47-b83ea8229c8a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.874234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311b920-bf72-43a8-8b8a-bea2a1f7f415 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:46.909225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.909941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.910227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.910507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] PATH_INFO: `/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.912259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] RBAC: Authorizing `identity:get_project(project_id=a1b5d6c51cda4ac682c8f2e91acd7d8c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.914024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.916663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 66/129] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.918994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.929769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:46.930427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:46.931186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:46.931186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:46.932296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:46.933866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:46.935770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 64/130] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:46.936278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:46.971763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.039895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db101-6ce1-44af-83aa-97d9681f2985 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.050693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b2c-6ce6-48fb-82b2-7f37e621a4b4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.081132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.083929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.084837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.085426 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.086023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.088103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.090016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.091182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.091822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.092107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.092395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.093306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 67/131] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:47.093528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.094436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:47.095075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.101484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 65/132] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:46 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:47.102356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:47.208139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef092cf9-caf2-4e02-9b34-aaf3acf7fcca None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.209548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374114d-46af-4f21-a8cf-b610602955e7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.219528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.246737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.247196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.247408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.247629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.247857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.247962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.248142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.248418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] PATH_INFO: `/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c/users/1a7f29f505094c969219499c619c7462/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.248633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=886c0bbbf0784048a18fb7be815dc421) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:47.248949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.250324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1b5d6c51cda4ac682c8f2e91acd7d8c, user_id=1a7f29f505094c969219499c619c7462, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.250420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.251790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.256934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 66/133] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:47 2022] GET /identity/v3/role_assignments?user.id=886c0bbbf0784048a18fb7be815dc421&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=8706fc9e7b654363a1548456569dc351 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:47.262830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:47.275093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 68/134] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:47 2022] PUT /identity/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c/users/1a7f29f505094c969219499c619c7462/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:47.276151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:47.411138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.480328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac478b-593a-4a48-bc7e-874f74c47d86 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.497397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72951ad1-c9ff-4572-bc92-891ba82acb34 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.521247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.522050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.522412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.522976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.525093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.526579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b70c0-e35e-4e17-a5ee-2a322420f79d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.528939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 67/135] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:47 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:47.534250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b01f5-164b-4fd7-8245-a034d04fa8b6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.534250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b01f5-164b-4fd7-8245-a034d04fa8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.534250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b01f5-164b-4fd7-8245-a034d04fa8b6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.534250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 68/136] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:47.535493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b01f5-164b-4fd7-8245-a034d04fa8b6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:47.545831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.546844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.547350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.547827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/75821779c93e4039bcee682de1f82886/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.550072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, user_id=75821779c93e4039bcee682de1f82886, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.552034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d045ae6-fa6f-48dd-9d78-58ef3804c21a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.568154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 69/137] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:47 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/75821779c93e4039bcee682de1f82886/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:47.570003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.570591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.571098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.592267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:47.593855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:47.621428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.676686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 70/138] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:47.748419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b01f5-164b-4fd7-8245-a034d04fa8b6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:47.787445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:47.799737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:47.800502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:47.800792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:47.801076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/1a7f29f505094c969219499c619c7462/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:47.803465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, user_id=1a7f29f505094c969219499c619c7462, role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:47.804978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371216ea-50b9-4b70-8cc7-33b927538d57 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:47.824266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 69/139] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:47 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/1a7f29f505094c969219499c619c7462/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:48.055999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:48.104501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46bd52-b3b0-4edc-9b59-a8c560447c85 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:48.104993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46bd52-b3b0-4edc-9b59-a8c560447c85 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:48.105066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46bd52-b3b0-4edc-9b59-a8c560447c85 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:48.106404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 70/140] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:23:48.113420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:48.113704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:48.113983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:48.140134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:48.141353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:48.224192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 71/141] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:48.240807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af92013-4774-4b00-9f68-e3a4510d6c5e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:48.286161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:48.287186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:48.287703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:48.288180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:48.289512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:48.291202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:48.292989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 71/142] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:48.298913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:48.450456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0e445-e39f-4b85-a8d9-40075b391794 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:48.469125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:48.489269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:48.489891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:48.490178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:48.490457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:48.491155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:48.492616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:48.523507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 72/143] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:23:48 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:48.709462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:48.750094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:48.847088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166ab25-d2ae-4d3c-a377-72267a5b7cb4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:48.886054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:48.886712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:48.887002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:48.887276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:48.903699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:48.909207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:48.910084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:48.910872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:48.914744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 72/144] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:48.916124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:49.057827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95e531-fc5d-44ce-98e0-1871c4a42bb4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:49.095673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:49.096547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:49.096834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:49.097116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:49.097793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:49.098100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:49.099518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:49.108100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 73/145] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:49.113812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:49.226919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:49.246098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7b231-f93b-4869-8dff-7b0a417a3701 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:49.286902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:49.287887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:49.288394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:49.288878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:49.290237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:49.291919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:49.294115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 73/146] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:49 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:49.302505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:49.362744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:49.442233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b8899-29c1-4f7b-8594-2e308e714e17 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:49.478706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:49.479713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:49.480179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:49.480636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:49.560499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:49.562287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:49.599116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:49.668999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 74/147] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:49.675683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:49.810946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d9567-abe4-4cdf-9b35-50dabbebd123 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:49.853863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:49.854428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:49.854732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:49.855010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:49.856175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:49.857630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d89a48-fcb8-4545-a28d-75efeebf1482 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:49.859548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 74/148] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:49 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:49.865663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:49.998322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158294e-9f90-49d9-9b1d-6d07128bffe5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:50.038213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:50.039199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:50.039720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:50.040192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:50.041124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:50.041634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:50.043317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde73b4-51a6-496c-a352-11e3276e6704 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:50.049798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:50.055232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 75/149] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:49 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:50.176749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be0cb-474a-4b96-9839-15967b849257 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:50.177227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be0cb-474a-4b96-9839-15967b849257 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:50.177227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7be0cb-474a-4b96-9839-15967b849257 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:50.178774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 75/150] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:50.184200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:50.185480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:50.185966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:50.186754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d491-5078-495c-bfff-b4215b9e645a None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:50.187048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d491-5078-495c-bfff-b4215b9e645a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:50.187329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d491-5078-495c-bfff-b4215b9e645a None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:50.188835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 76/151] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:50.194545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:50.195093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:50.195383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:50.221303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:50.222535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:50.225696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:50.227025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:50.261699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:50.351664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 77/152] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:50.356892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 76/153] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:50.845133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:50.916221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.012953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10425f-ff4e-4a92-a342-6346706e5d14 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:51.078938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:51.079930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.080879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.081298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.093853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-6442a636-1d83-4994-88fe-98e8754a7923 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:51.101180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:51.102310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:51.103318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:51.104037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 78/154] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:51.105117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aaf38-d154-4d61-9498-29d1ce58e3d2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.105568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aaf38-d154-4d61-9498-29d1ce58e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.106052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aaf38-d154-4d61-9498-29d1ce58e3d2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.107715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 79/155] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:51.108980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d8-4f36-4c36-85ad-7e78288c6457 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.109509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d8-4f36-4c36-85ad-7e78288c6457 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.109968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d8-4f36-4c36-85ad-7e78288c6457 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.113892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 80/156] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:51.114952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f93d8-4f36-4c36-85ad-7e78288c6457 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.131058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81963e17-29dd-40c7-a4af-b21b869b4b68 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:51.148462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:51.163012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:51.171574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:51.172591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.173038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.173656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.186462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-03d94362-e09d-4248-93c0-b180a6cfc0b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:51.193307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:51.194360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:51.195520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:51.198032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 77/157] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:23:51.200208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316889-40bd-4e76-add0-457af48ec445 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.200687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316889-40bd-4e76-add0-457af48ec445 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.201134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316889-40bd-4e76-add0-457af48ec445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.224649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c0316889-40bd-4e76-add0-457af48ec445 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:51.226237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0316889-40bd-4e76-add0-457af48ec445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:51.307357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 78/158] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:51.329106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.329637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.330066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.362442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:51.364542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:51.384851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f93d8-4f36-4c36-85ad-7e78288c6457 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:51.466887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:51.467598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.467834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.468149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.468827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:51.469128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:51.470566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:51.486728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 81/159] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:51.487647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.520139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 79/160] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:51.520824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.617843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:51.645266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:51.672016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f669d-c54b-4dcc-a918-fd417990d2cb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:51.744601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:51.745357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.745644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.745923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.746605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:51.746941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:51.748383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:51.765165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 82/161] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:51.774524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.794562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964d10a-f8bb-48af-a661-c43cd3cf6342 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:51.883591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:51.884170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:51.884449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:51.884719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] PATH_INFO: `/v3/users/886c0bbbf0784048a18fb7be815dc421` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:51.886051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] RBAC: Authorizing `identity:get_user(user_id=886c0bbbf0784048a18fb7be815dc421)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:51.887576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:51.889294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 80/162] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:51 2022] GET /identity/v3/users/886c0bbbf0784048a18fb7be815dc421 => generated 289 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:51.896417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:51.965739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0b9f2-bedd-4a83-93a5-65132ef364f5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.007500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.008524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.009035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.009521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.011041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.012882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.014809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 83/163] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:51 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:52.015884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.020872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.124154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d73d5-9171-4ac3-bdfa-a71c82827237 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.133512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371afcd5-8823-4dd6-872d-9c07fdabceae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.165466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.166700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.167320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.167817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.173095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.180619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:52.186344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:52.187306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:52.188088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:52.191340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 84/164] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:52.192476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.220654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.221861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.222187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.222604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.227363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.228764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.230564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 81/165] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:51 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:52.231642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.266683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.308944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2860f-f6f7-4b1a-ba02-3606952c41a7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.347516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.348552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.349037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.349513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.378715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9b661-35e6-4771-aaaf-55cdfd43d4ee None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.424832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.427049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.447395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352625-b789-499a-abe0-b7b80886139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.449671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352625-b789-499a-abe0-b7b80886139d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.449671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352625-b789-499a-abe0-b7b80886139d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.449671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352625-b789-499a-abe0-b7b80886139d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.464081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352625-b789-499a-abe0-b7b80886139d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-50352625-b789-499a-abe0-b7b80886139d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:52.469568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:52.470471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:52.471388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 82/166] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:52.472057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352625-b789-499a-abe0-b7b80886139d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.503476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.529149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.546408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 85/167] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:52.547427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.589615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352625-b789-499a-abe0-b7b80886139d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.628252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.629260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.629684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.630114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.631257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:52.631717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.633364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ae4ce-8f58-4647-84b1-d70668ade54f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.642250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 83/168] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:52.648854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3afe67-052b-4b08-91b8-d0f89fa29628 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.649366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3afe67-052b-4b08-91b8-d0f89fa29628 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.649821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3afe67-052b-4b08-91b8-d0f89fa29628 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.651566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 84/169] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:52.652478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3afe67-052b-4b08-91b8-d0f89fa29628 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.656804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b109f1-d02b-4d07-ab74-db3c8e166dc2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.692384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.693146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.693500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.693889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/886c0bbbf0784048a18fb7be815dc421/roles/8706fc9e7b654363a1548456569dc351` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.696135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, user_id=886c0bbbf0784048a18fb7be815dc421, role_id=8706fc9e7b654363a1548456569dc351)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.698393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.714876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 86/170] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:52 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/users/886c0bbbf0784048a18fb7be815dc421/roles/8706fc9e7b654363a1548456569dc351 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:52.719013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.778779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.855169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3afe67-052b-4b08-91b8-d0f89fa29628 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.873778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:52.921233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.922215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.922892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.923522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.924527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:52.925040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.926837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.928338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26c71a-83b3-4fef-81b7-40cf6681ad0b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:52.937631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 85/171] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:52.938441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:52.976318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:52.977157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.978310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.978591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] PATH_INFO: `/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:52.979793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] RBAC: Authorizing `identity:get_project(project_id=a1b5d6c51cda4ac682c8f2e91acd7d8c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:52.981248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8339c-b9b8-40af-8f15-9a4659c54065 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:52.983502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 87/172] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/projects/a1b5d6c51cda4ac682c8f2e91acd7d8c => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:52.985151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfeacd-9c3c-4d62-8bb4-84000b012901 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:52.985437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfeacd-9c3c-4d62-8bb4-84000b012901 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:52.985735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfeacd-9c3c-4d62-8bb4-84000b012901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.010701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3dbfeacd-9c3c-4d62-8bb4-84000b012901 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:53.012202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3dbfeacd-9c3c-4d62-8bb4-84000b012901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:53.045261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbfbfe-2df5-47d3-acb1-10060dd6264a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.084413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.085214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.085492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.085803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] PATH_INFO: `/v3/users/1a7f29f505094c969219499c619c7462` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.086665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 88/173] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:53.086977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] RBAC: Authorizing `identity:get_user(user_id=1a7f29f505094c969219499c619c7462)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:53.087933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc32b8-0d38-43b8-85da-39f3f3086803 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.088229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc32b8-0d38-43b8-85da-39f3f3086803 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.088432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c15fd-8198-4391-ab76-824af29a5c43 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:53.088525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc32b8-0d38-43b8-85da-39f3f3086803 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.089744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 89/174] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:53.090447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc32b8-0d38-43b8-85da-39f3f3086803 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.094688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 86/175] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:52 2022] GET /identity/v3/users/1a7f29f505094c969219499c619c7462 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:53.095726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190fc61-5d6f-4ca5-8013-ae168aac3846 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.096011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190fc61-5d6f-4ca5-8013-ae168aac3846 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.096879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190fc61-5d6f-4ca5-8013-ae168aac3846 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.102157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 87/176] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:53.102157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190fc61-5d6f-4ca5-8013-ae168aac3846 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.127088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.155364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.216305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc32b8-0d38-43b8-85da-39f3f3086803 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.250162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190fc61-5d6f-4ca5-8013-ae168aac3846 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.255888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.256737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.257171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.257632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] PATH_INFO: `/v3/users/75821779c93e4039bcee682de1f82886` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.259040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] RBAC: Authorizing `identity:get_user(user_id=75821779c93e4039bcee682de1f82886)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:53.260748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69fe82-702c-4822-a343-36343cbc40aa None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:53.262973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 90/177] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/users/75821779c93e4039bcee682de1f82886 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:53.264457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf14b-5536-4869-adaa-c66a14c22bd3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.264947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf14b-5536-4869-adaa-c66a14c22bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.265411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf14b-5536-4869-adaa-c66a14c22bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.285552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b87cf14b-5536-4869-adaa-c66a14c22bd3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:53.287441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b87cf14b-5536-4869-adaa-c66a14c22bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:53.289173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.289776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.290083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.290671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.291395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=a1b5d6c51cda4ac682c8f2e91acd7d8c, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:53.292552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:53.293999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:53.306699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 88/178] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=a1b5d6c51cda4ac682c8f2e91acd7d8c&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:53.306699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.407141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 91/179] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:53.409006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.409590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.410045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.439473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04d1bb-32de-4ce1-9122-ab047a5a2177 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.441881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:53.444155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:53.444953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.476545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.510789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.511337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.511616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.511898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.538501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 92/180] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:53.539191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.583818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:53.585310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:53.644568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d8ab4-7c2e-470d-bd7a-289256bcc5f9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.679715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.680331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.680609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.680878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.701635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.714176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 89/181] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:53.715077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.716170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.761154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:53.762547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:53.852842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98e16-c245-4030-aa90-3fcc4801de71 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:53.868887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 93/182] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:53.869553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.900097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:53.901766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:53.902263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:53.903087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:53.918017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:53.925977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:53.926876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:53.927703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:53.928960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 90/183] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:53.928960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:53.936543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:53.991948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eadf9-8e8d-4f2c-89c1-2419fab1a449 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.015030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.030148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.030807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.031091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.031361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.032503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.034077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.040381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 94/184] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.040381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.062983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7222f-383c-46e2-8539-9ef6718ab1e0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.099966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.100596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.100878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.101155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.102505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.103995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.105817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 91/185] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.107516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.146835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8433d4b-b90b-4625-83ef-bbbb3f8e2179 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.165459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.184935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.185647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.186072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.186417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.187710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.189117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.191652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 95/186] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.192157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.226042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcd00e-46ac-4bf3-a3cb-b0bdbfecb943 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.254702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.263881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.264487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.264784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.265115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.265731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:54.266040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.267472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.278685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 92/187] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.278685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.302501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b023-b1d7-4f8f-92a5-d55a83f61206 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.345794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.347217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.347637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.348222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.359517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:54.364809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:54.365905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 96/188] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:54.366936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.395338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642d40c-4dfc-49dc-9e5c-773d7bb0960a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.417688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.435792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.436704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.436987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.437259 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.438045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=1a7f29f505094c969219499c619c7462) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:54.438360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.440012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.448950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 93/189] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/role_assignments?user.id=1a7f29f505094c969219499c619c7462&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.451536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.482607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7de611-110e-4e7d-b714-4fab72989ffa None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.523643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.524379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.524711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.525093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.526021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.527624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.528032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.553038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 97/190] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:23:54 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:54.554149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.588779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a1d9-7f09-4e9c-ba56-6c3924831f41 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.633387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.634251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.634703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.635147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.636062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=75821779c93e4039bcee682de1f82886) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:54.636532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.638141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.645887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 94/191] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/role_assignments?user.id=75821779c93e4039bcee682de1f82886&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.647078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.690238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a6a1-b4e2-4a88-bef4-79d3248141bb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.691705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.739517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.740355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.741540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.742590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.766208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.768803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.771035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 98/192] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.777135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.816898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805f597-7250-4bde-9be7-917af6ab44e3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.842603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:54.867731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.868654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.869079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.869614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.870480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:54.871052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:54.872715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcf27b-da5f-40d0-92c0-1864ac410b12 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:54.884569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 95/193] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:54.886170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624af5c-a494-4b05-8cb6-8bcceac0bda2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.886604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624af5c-a494-4b05-8cb6-8bcceac0bda2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.887054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624af5c-a494-4b05-8cb6-8bcceac0bda2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.888449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 96/194] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:54.890331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624af5c-a494-4b05-8cb6-8bcceac0bda2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:54.897374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65bf76-29fc-4f93-94f4-17604ed1575b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:54.942993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:54.943647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:54.943931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:54.944206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:54.982451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.022286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624af5c-a494-4b05-8cb6-8bcceac0bda2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.032660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.034848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.066115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.066891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.067176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.067453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.068657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.070124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eea05-606e-4abc-886c-0a280172957e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.071921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 97/195] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:55.072878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.073162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.073435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.093444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:55.094657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:55.162038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.165840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 99/196] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:23:55.167104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.176712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 98/197] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:55.178047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.250682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.302539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414451f8-c446-41fa-8966-211bdd0dcb7d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.332562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27713e31-8a0f-4476-a7ed-751a1520879e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.344211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.345148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.345641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.346104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.348434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.350535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.352839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 99/198] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:55.362718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.401390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.402011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.402287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.402559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.439037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.517855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62864-d71b-4d06-a671-5bbde3b8d09a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.539257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.540752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.565013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.566958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.567728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.568437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.569350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=801aa9fe4a1548ef9dd1975a27528659) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:55.569963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.572199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.579662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 100/199] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/role_assignments?user.id=801aa9fe4a1548ef9dd1975a27528659&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:55.580697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.649670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 100/200] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:55.650135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.656303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.778809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d2471-2c42-45ff-aca2-d0caffee7ea0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.808449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:55.843649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3319b-7987-4016-8998-3f472f46f354 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:55.884257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.885271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.885753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.886234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.887910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:55.889636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:55.891633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 101/201] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:55.891895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:55.897042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:55.897042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:55.897042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:55.898398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.926220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:55.931697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:55.932725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:55.934142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 101/202] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:55.942345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:55.942803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:56.031328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd8173-97bd-4c7e-bece-6ab625c08b0a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:56.075313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:56.076772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:56.077340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:56.077821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:56.079094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=801aa9fe4a1548ef9dd1975a27528659) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:56.079930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:56.082733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:56.089482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 102/203] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/role_assignments?user.id=801aa9fe4a1548ef9dd1975a27528659&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:56.130970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcd76-d3cb-447c-a4c9-36ddbdcb6b10 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:56.168232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:56.169211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:56.169693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:56.170191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:56.171117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:56.171642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:56.173509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:56.183966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 102/204] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:56.189963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:56.355225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb84d6-cfe1-4081-81f7-2d6ceeaf295c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:56.358958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:56.409052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:56.409885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:56.410324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:56.410739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] PATH_INFO: `/v3/users/886c0bbbf0784048a18fb7be815dc421` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:56.412001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] RBAC: Authorizing `identity:get_user(user_id=886c0bbbf0784048a18fb7be815dc421)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:56.413560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:56.415794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 103/205] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:56 2022] GET /identity/v3/users/886c0bbbf0784048a18fb7be815dc421 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:56.423714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:56.480231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:56.608771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6389-8715-48ed-806c-6ec1fa7b6a86 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:56.688636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:56.688636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:56.688636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:56.688636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:56.815978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:56.818120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:56.869823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:56.999017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 103/206] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:57.004885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:57.171614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af60f21-3d44-4618-be66-4b935a4d17ad None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:57.216702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:57.217599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.217882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.218155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.219322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:57.220767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6b16c-c8cd-4ac7-b787-aef35a0dca2d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:57.223959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 104/207] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:57 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:57.231749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:57.382810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721695-66b3-4eed-8d33-4f938cd1312d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:57.421730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:57.433778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:57.435202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.435698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.436161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.437079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, scope.project.id=e0dfcf0f119a48eebacae56532f7a756, user.id=886c0bbbf0784048a18fb7be815dc421) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:57.437579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:57.439231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6c222-b6a3-4c00-be1a-46397722ba42 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:57.447392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 104/208] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:57 2022] GET /identity/v3/role_assignments?user.id=886c0bbbf0784048a18fb7be815dc421&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=8706fc9e7b654363a1548456569dc351 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:57.448685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81fd0-6faa-40c8-9940-ab6912466b88 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.448982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81fd0-6faa-40c8-9940-ab6912466b88 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.449256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81fd0-6faa-40c8-9940-ab6912466b88 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.450730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 105/209] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:57.463761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.464072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.464325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.492462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:57.493668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:57.511474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6cc30-2d85-4c5e-bb61-059201be43eb None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.512096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6cc30-2d85-4c5e-bb61-059201be43eb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.512575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6cc30-2d85-4c5e-bb61-059201be43eb None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.515408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 106/210] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:57.520596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:57.521121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:57.521618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:57.540789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:57.542378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:57.568333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:57.591486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 105/211] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:57.650781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 107/212] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:57.920567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:57.938235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.044916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7651-bbf4-4ebf-a012-abc372c3894c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.076042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac89ec6-1720-4580-a692-cda83b92b70d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.079292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.085690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.086336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.086622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.086994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.099073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-bd0b08d8-3f35-4179-954b-c39f9e6517d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:58.105973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:58.106947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 106/213] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:58.112100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f032-edd5-41f0-b966-332c1ace6816 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.113615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f032-edd5-41f0-b966-332c1ace6816 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.113615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350f032-edd5-41f0-b966-332c1ace6816 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.113615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 107/214] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:58.113795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.114480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.115678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350f032-edd5-41f0-b966-332c1ace6816 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.116329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.116606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.117744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.119230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87152ce6-8e3a-4eb1-a2cb-aa10c5621c59 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.125545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 108/215] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:58.125545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.125545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.125545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.141684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:58.142904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:58.165975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.226709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 109/216] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:23:58.227601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.230920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350f032-edd5-41f0-b966-332c1ace6816 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.266845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.267454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.267730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.267999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.268805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:58.269801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.271528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.280583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 108/217] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:58.286455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.358487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d022ab-4833-4ad4-b352-b3451bf485e9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.363525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.396099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.396731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.396999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.397310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.397966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.399473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0fb90-f02b-42c4-ab8c-a9a1a5521a6c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.409841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a9bb-3ad7-4b1c-bb01-e8292fe09b72 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.421357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 110/218] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:23:58 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:23:58.422518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a131d3-2630-46a1-8071-ff5140ce3de4 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.422839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a131d3-2630-46a1-8071-ff5140ce3de4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.423130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a131d3-2630-46a1-8071-ff5140ce3de4 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.424469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 111/219] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:23:58.430497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.430825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.431167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.454099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:23:58.455554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:23:58.467005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.467560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.469548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.469857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.470137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] PATH_INFO: `/v3/users/21369df04f1e4551b886c336b5b59e14` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.486705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] RBAC: Authorizing `identity:get_user(user_id=21369df04f1e4551b886c336b5b59e14)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.488871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.491181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 109/220] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/users/21369df04f1e4551b886c336b5b59e14 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:58.492264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.549624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 112/221] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:23:58.551343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.615250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a1c6-4b70-4fd4-92e7-4f88d2477834 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.623103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.663508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.664151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.664429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.664703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.677551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:58.682447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba528b6-9e8d-4698-a01e-0c40bdffb557 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:58.682588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:58.683454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:23:58.684249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:58.685419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 110/222] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:23:58.692877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.729507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.730124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.730402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.730940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.752393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.830100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab652-0e82-42b9-a52f-31ce7c97f4f6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:58.831328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.833125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.877797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089031-5a76-4288-a65a-582e510a485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:58.879505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089031-5a76-4288-a65a-582e510a485e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:58.880019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089031-5a76-4288-a65a-582e510a485e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:58.880483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089031-5a76-4288-a65a-582e510a485e None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:58.881440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52089031-5a76-4288-a65a-582e510a485e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:58.881943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52089031-5a76-4288-a65a-582e510a485e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:58.884559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52089031-5a76-4288-a65a-582e510a485e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:58.896738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 111/223] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:23:58.897598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089031-5a76-4288-a65a-582e510a485e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:58.930731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:58.963320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 113/224] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:23:58.964629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.010974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.011139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089031-5a76-4288-a65a-582e510a485e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.053033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.053977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.054088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.054330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.065999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:23:59.071904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:23:59.072802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:23:59.073607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 112/225] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:23:59.074273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.076106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd88ed3-2a8b-4d80-a56b-8854cd930b55 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.118857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.120044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.120529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.121004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.122441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.124397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.130849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 114/226] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:58 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.132065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.172843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.186190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e589c0-3141-417f-8e5f-faf51056c70c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.221992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.222792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.223169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.223446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.224740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.226195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.228306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 113/227] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.228843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.248011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0491a0-dcbe-4063-b623-9d6ef397230f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.259661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.285342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.286012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.286399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.286831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.287622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:59.288030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.289499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.298139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 115/228] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.299161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.356336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5eac78-c306-475f-8f24-c1dcb34a196b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.400285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.401510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.401871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.402271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.403798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.405998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.407748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 114/229] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.411518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.425479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fde668-fad9-4049-9351-e5a574e7058f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.446982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.461931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.462938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.463419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.463887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.464867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=21369df04f1e4551b886c336b5b59e14) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:23:59.465389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.467228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.474054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 116/230] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/role_assignments?user.id=21369df04f1e4551b886c336b5b59e14&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.475144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.517903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.608846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b3782-5338-4f71-a87e-d77a29adcd53 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.616544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e0314-6af4-47cb-97ca-8ba470c4b805 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.664775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.665827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.666306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.666408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.666947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/801aa9fe4a1548ef9dd1975a27528659/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.667050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.667344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.667621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.670008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.670321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.672062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 115/231] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.672928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, user_id=801aa9fe4a1548ef9dd1975a27528659, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.674477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.677819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.706553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 117/232] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:59 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/801aa9fe4a1548ef9dd1975a27528659/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:59.804367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:59.813737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32529892-7d22-4e49-9994-62dc61250ed5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:23:59.852396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:23:59.853047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:23:59.853330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:23:59.853608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:23:59.854816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:23:59.856270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf681e-619c-46c9-b2be-b498423b48d8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:23:59.858224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 116/233] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:23:59 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:23:59.864924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:23:59.997503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:00.004932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8be486-4d77-4ef8-915f-6a68c45fcb66 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:00.055387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:00.056073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:00.056355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:00.056632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/801aa9fe4a1548ef9dd1975a27528659/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:00.058846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, user_id=801aa9fe4a1548ef9dd1975a27528659, role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:00.060321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0db574-0a83-43ea-9620-0905e3158375 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:00.091409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 118/234] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:23:59 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/801aa9fe4a1548ef9dd1975a27528659/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:00.837064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a98b03-7bba-4a87-b9de-010292e59aaa None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:00.837581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a98b03-7bba-4a87-b9de-010292e59aaa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:00.837656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a98b03-7bba-4a87-b9de-010292e59aaa None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:00.839131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 117/235] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:00.847403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213dfc-3ac5-4b9d-a46b-bdbdd40c6bae None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:00.847696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213dfc-3ac5-4b9d-a46b-bdbdd40c6bae None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:00.847982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14213dfc-3ac5-4b9d-a46b-bdbdd40c6bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:00.875952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-14213dfc-3ac5-4b9d-a46b-bdbdd40c6bae None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:00.877167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-14213dfc-3ac5-4b9d-a46b-bdbdd40c6bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:00.967115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8beafc-10a2-4da5-9f3f-ac2da86ce38a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:00.967303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 119/236] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:00.967379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8beafc-10a2-4da5-9f3f-ac2da86ce38a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:00.967925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8beafc-10a2-4da5-9f3f-ac2da86ce38a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:00.969348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 118/237] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:00.974574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e11e8-9074-4d68-893a-547559b53465 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:00.974887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e11e8-9074-4d68-893a-547559b53465 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:00.975157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e11e8-9074-4d68-893a-547559b53465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.001773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-805e11e8-9074-4d68-893a-547559b53465 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:01.002977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-805e11e8-9074-4d68-893a-547559b53465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:01.083465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 120/238] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:01.335479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8beafc-10a2-4da5-9f3f-ac2da86ce38a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:01.478334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8beafc-10a2-4da5-9f3f-ac2da86ce38a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:01.499476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e11e8-9074-4d68-893a-547559b53465 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:01.523831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:01.546072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:01.546735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:01.547020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:01.547295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.548419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:01.549971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:01.552699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 119/239] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:01.558428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:01.616980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:01.685183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e11e8-9074-4d68-893a-547559b53465 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:01.754296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c3e85-1c70-41e1-8549-af8a88049707 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:01.757077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:01.757719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:01.757995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:01.758263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.770298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ba28f893-99e1-4e9f-a027-078581ec80fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:01.775523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:01.776425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:01.777235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 121/240] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:01.777891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33baf0-cc8c-4230-83e8-3bb4753199ed None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:01.778721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33baf0-cc8c-4230-83e8-3bb4753199ed None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:01.778721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33baf0-cc8c-4230-83e8-3bb4753199ed None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.783725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 122/241] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:01.788350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33baf0-cc8c-4230-83e8-3bb4753199ed None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:01.805556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:01.806628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:01.807176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:01.807645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.809450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:01.811418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671062-b121-405d-bbaf-578ad3adbad7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:01.837623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 120/242] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:01 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:01.839798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189856c4-50e8-4466-a934-a037fe487701 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:01.840301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189856c4-50e8-4466-a934-a037fe487701 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:01.840822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189856c4-50e8-4466-a934-a037fe487701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:01.883868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-189856c4-50e8-4466-a934-a037fe487701 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:01.886076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-189856c4-50e8-4466-a934-a037fe487701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:01.922042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:01.973737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33baf0-cc8c-4230-83e8-3bb4753199ed None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.027641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 121/243] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:02.035625 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.036241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.036566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.036835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.037505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:02.037806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:02.039275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:02.052074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 123/244] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:02.055100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189856c4-50e8-4466-a934-a037fe487701 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.072943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.164207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189856c4-50e8-4466-a934-a037fe487701 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.200709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.201345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.201622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.201896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] PATH_INFO: `/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.218512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f5a10ef3ce48b8a1664802e700b3a6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:02.220010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a5ecc-068e-4317-82d0-6840f6d11e15 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:02.225225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 122/245] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:02.228190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.344900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305e262-4eb8-4789-a58b-4b44065ec6ba None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.355853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f85b3-ee53-4348-b617-b55a86414c10 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.356131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f85b3-ee53-4348-b617-b55a86414c10 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.356520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f85b3-ee53-4348-b617-b55a86414c10 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.357866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 123/246] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:02.363532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.363801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.364071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.384369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:02.385709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:02.391227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.391925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.392201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.392472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.404067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.477050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:02.477879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 124/247] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:02.478512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e8766-8981-4c17-8cba-4ac44248c0ee None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:02.478760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.481354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.613003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31e403-81e4-40e3-b25b-5145dcee62b5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.613847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 124/248] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:02.615412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27503c-171b-4e31-942d-96f807682816 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.616585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27503c-171b-4e31-942d-96f807682816 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.616954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27503c-171b-4e31-942d-96f807682816 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.618319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 125/249] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:02.622950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27503c-171b-4e31-942d-96f807682816 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.670790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.671434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.671713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.671986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.672618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.691904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2d7da914-7c7c-4c35-bb33-48c0cbb0f772 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:02.697298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:02.698174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:02.698993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 125/250] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:02.700406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.700406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.700406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.735734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:02.736852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:02.742442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27503c-171b-4e31-942d-96f807682816 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.778311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.778982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.779263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.779652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.780811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:02.782262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:02.784311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 126/251] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:02.785804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.840380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.853657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 126/252] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:02.854745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:02.900499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f409-f22e-4527-866c-e35152e38587 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:02.908820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:02.940614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:02.942151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:02.942924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:02.943614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:02.944994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:02.945478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:02.947341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:02.958413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 127/253] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:02.959539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.068289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7a52d-0d58-4862-b4e0-56dee874e032 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.077809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea099f13-c9a0-4825-b08c-3d8518987698 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.106191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.107221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.107807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.108286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.124025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734313-7067-420f-9903-ae4814220121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.124635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734313-7067-420f-9903-ae4814220121 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.124917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734313-7067-420f-9903-ae4814220121 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.125192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734313-7067-420f-9903-ae4814220121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.125924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3734313-7067-420f-9903-ae4814220121 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=f3f5a10ef3ce48b8a1664802e700b3a6) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:03.126239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3734313-7067-420f-9903-ae4814220121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.127741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3734313-7067-420f-9903-ae4814220121 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.133626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 127/254] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/role_assignments?user.id=f3f5a10ef3ce48b8a1664802e700b3a6&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=8706fc9e7b654363a1548456569dc351 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.134617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734313-7067-420f-9903-ae4814220121 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.138156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:03.144018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:03.145525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:03.146328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 128/255] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:03.153217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.216414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:03.227327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:03.303458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9084bb-a88e-4967-ae35-6fb6c3afcbfc None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.320372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734313-7067-420f-9903-ae4814220121 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.341067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.341845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.342227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.342605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.343668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:03.344073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.345596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.355885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 129/256] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.361295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.392053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.392704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.392985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.393263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] PATH_INFO: `/v3/users/21369df04f1e4551b886c336b5b59e14` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.395660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] RBAC: Authorizing `identity:get_user(user_id=21369df04f1e4551b886c336b5b59e14)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.397129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.398851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 128/257] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/users/21369df04f1e4551b886c336b5b59e14 => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.399621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.482051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:03.492824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf218b8-8921-4d0e-a5d7-2d88cd999236 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.532541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac28e-d9bc-423a-b876-162d447d8630 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.544478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.544874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.545162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.545439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.559681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:03.566540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:03.571980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:03.572220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:03.573063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:03.573874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 130/258] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:03.574538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.580392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.581055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.581568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.583859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.585585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.587452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 129/259] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.588479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.703271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53766e-a14d-4033-9fd9-6eac4ebaf018 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.743077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.744211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.744726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.745377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.746386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:03.746941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.748641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.762753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 130/260] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.763964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.824488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:03.862379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81dd2d-8964-490c-87ee-34dc77fc3020 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.886965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcf5af-c5ed-4760-8b25-c1ed51dd3248 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:03.926047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.926770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.927078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.927339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.930429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:03.931361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:03.931841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:03.932326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:03.936904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:03.938741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:03.940803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 131/261] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:03.942302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:03.969951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.002294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.004483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.075167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.120904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e57c0-de5d-4632-b5fb-acf2891084b9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:04.159431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:04.160641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:04.160760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 131/262] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:04.160946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:04.161224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] PATH_INFO: `/v3/users/801aa9fe4a1548ef9dd1975a27528659` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:04.161554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:04.166901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=801aa9fe4a1548ef9dd1975a27528659)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.168944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.170593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 132/263] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:03 2022] GET /identity/v3/users/801aa9fe4a1548ef9dd1975a27528659 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:04.171377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:04.279991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.295823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f98f81-d790-40fe-a3d0-3cce9ed2e7e6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:04.299874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398271f-1661-40f4-8487-482f57f6545d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:04.331489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:04.332098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:04.332380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:04.332657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:04.333800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.335279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.337101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 133/264] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:04 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:04.337673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:04.356792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:04.357472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:04.357755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:04.358033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/21369df04f1e4551b886c336b5b59e14/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:04.360120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, user_id=21369df04f1e4551b886c336b5b59e14, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.361774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.378448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 132/265] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:04 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/21369df04f1e4551b886c336b5b59e14/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:04.380047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:04.398689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.532618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb8b1e-2a6a-4d95-8eb7-da1349de4b60 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:04.569636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:04.570615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:04.571124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:04.571652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:04.604131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524649a-2e9c-4ec5-b48d-560a45029c92 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:04.607393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.655193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.656999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275fbb6-e10e-4824-a2a1-cee642ad5bb0 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.682083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:04.685752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:04.686240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:04.686739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:04.687685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=801aa9fe4a1548ef9dd1975a27528659) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:04.689188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:04.690941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:04.702296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 133/266] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:04 2022] GET /identity/v3/role_assignments?user.id=801aa9fe4a1548ef9dd1975a27528659&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:04.814957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 134/267] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:04.820520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:04.925354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:04.957050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698614f3-4940-455f-962d-3bee27c4fb08 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:05.012210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:05.013203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.013706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.015515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00748da-b491-441b-b157-d733e8df71c9 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.016352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00748da-b491-441b-b157-d733e8df71c9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.016838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00748da-b491-441b-b157-d733e8df71c9 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.018388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 135/268] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:05.018816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.020465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:05.022190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:05.024063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ed8d-61e6-424e-9c30-2d9b80cd8b8f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.024351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ed8d-61e6-424e-9c30-2d9b80cd8b8f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.024709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ed8d-61e6-424e-9c30-2d9b80cd8b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.025111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 134/269] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:04 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:05.030354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:05.067737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4411ed8d-61e6-424e-9c30-2d9b80cd8b8f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:05.068983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4411ed8d-61e6-424e-9c30-2d9b80cd8b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:05.145263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:05.176806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 136/270] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:05.178870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c303ee-4e03-41e6-8717-9527a91b0bcc None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.179305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c303ee-4e03-41e6-8717-9527a91b0bcc None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.179753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c303ee-4e03-41e6-8717-9527a91b0bcc None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.181185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 137/271] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:05.189285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.189880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.190416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.196624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454f736-173e-431a-9e04-8a4014c1ea05 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:05.227485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:05.228680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:05.266265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:05.266904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.267182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.267460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.268190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=801aa9fe4a1548ef9dd1975a27528659) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:05.268501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:05.271252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:05.287245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 135/272] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:05 2022] GET /identity/v3/role_assignments?user.id=801aa9fe4a1548ef9dd1975a27528659&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:05.359355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 138/273] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:05.583366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:05.597017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:05.684872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:05.701396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc596-6dfc-4fb8-a6f4-a6b5fd29f32c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:05.748407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:05.749477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.749766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.750189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.751722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:05.753609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c1a71-cb05-4829-93b8-5994d5b347fb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:05.756407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 136/274] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:05.757891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b84e-4463-4787-adf8-ed72c3527317 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.758181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b84e-4463-4787-adf8-ed72c3527317 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.758457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486b84e-4463-4787-adf8-ed72c3527317 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.763667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 137/275] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:05.764419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b84e-4463-4787-adf8-ed72c3527317 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:05.808248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de446b2-ab19-4ffe-858b-e41f67c3d2e8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:05.823628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:05.846708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:05.847270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.847556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.847959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.859000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:05.859699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-d7624207-6dfd-448b-99cb-27513a876718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:05.864262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:05.865284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:05.866088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 139/276] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:05.867353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.867353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.867353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.885594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:05.886836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:05.890584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486b84e-4463-4787-adf8-ed72c3527317 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:05.926433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:05.927369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:05.927717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:05.928068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:05.928722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:05.930170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:05.954982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 138/277] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:05 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:05.959645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:05.967975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 140/278] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:06.086668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaa3b6-d4db-478e-ac8b-e2a5f6d1066c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.126310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.126876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.127218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.127499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.128178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:06.128606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:06.130924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:06.132630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:06.143308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 139/279] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:06.148928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:06.245431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:06.261756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ff33-d807-486f-946e-4c84be14c653 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.265574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:06.309810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.310617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.311116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.311309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] PATH_INFO: `/v3/users/031d4b528373469fa9e14131bff16952` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.332085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] RBAC: Authorizing `identity:get_user(user_id=031d4b528373469fa9e14131bff16952)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:06.334137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:06.336399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 141/280] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/users/031d4b528373469fa9e14131bff16952 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:06.342575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:06.396046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058cf1f-4079-481e-ba41-f4963acbfba5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.444286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.445102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.445487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.445861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.463310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:06.464035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572aa14-6253-47f3-b3b6-418256c3c0da None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:06.470014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:06.470999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:06.471787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 140/281] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:06.480491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:06.531929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.533073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.534724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.536352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.537717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, user.id=031d4b528373469fa9e14131bff16952, scope.system=all) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:06.538891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:06.540749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff15f3-e5ef-4aae-b690-ece0d85af91f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:06.551087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 142/282] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/role_assignments?user.id=031d4b528373469fa9e14131bff16952&scope.system=all&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:06.590954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:06.662768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15b17e-84f3-4718-bad3-134ffa1b77fc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.686316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb9d6-0910-4fac-86b9-05dedc39b8d7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.686678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb9d6-0910-4fac-86b9-05dedc39b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.686916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcb9d6-0910-4fac-86b9-05dedc39b8d7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.688307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 143/283] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:06.694582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.694884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.695160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.714621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:06.715848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:06.717917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.719051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.719525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.719985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.720865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:06.721416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:06.723172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:06.739798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:06.741677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 141/284] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:06.747584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:06.823859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 144/285] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:06.875181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7cd0-1c10-4499-9786-0129455510aa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:06.894249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:06.913901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:06.914461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:06.914773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:06.915050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] PATH_INFO: `/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:06.916191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f5a10ef3ce48b8a1664802e700b3a6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:06.917636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:06.919541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 142/286] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:06.925040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.044475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b775d9a-d014-4eb4-8fcf-feea25c7afef None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.079859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.080491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.080772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.081048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:07.082284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:07.084328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:07.086110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 145/287] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:06 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:07.091873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.195437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2af-025d-4942-8fb7-2e04d22e2539 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.230214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.231261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.231556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.231827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.232265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/f3f5a10ef3ce48b8a1664802e700b3a6/roles/8706fc9e7b654363a1548456569dc351` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:07.233845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, user_id=f3f5a10ef3ce48b8a1664802e700b3a6, role_id=8706fc9e7b654363a1548456569dc351)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:07.235516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:07.265995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 143/288] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:07 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/users/f3f5a10ef3ce48b8a1664802e700b3a6/roles/8706fc9e7b654363a1548456569dc351 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:07.270527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:07.358553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f351c67-4f82-43ce-8f0a-5d2b898886d8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.404652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.405367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.405655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.405926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:07.418473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:07.428364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:07.429331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:07.430142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 146/289] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:07.439568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.507651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:07.569339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b8619-161a-40f2-8cc1-76a2b91e53f4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.606218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.606865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.607147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.607427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:07.608097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:07.608402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:07.609780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:07.618926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 144/290] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:07.624393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.737952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:07.758090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c75734-79d6-4c07-b5a4-52b594d88fe3 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.800710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.801941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.802260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.802502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] PATH_INFO: `/v3/users/21369df04f1e4551b886c336b5b59e14` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:07.803764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] RBAC: Authorizing `identity:get_user(user_id=21369df04f1e4551b886c336b5b59e14)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:07.805335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:07.807136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 147/291] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:07 2022] GET /identity/v3/users/21369df04f1e4551b886c336b5b59e14 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:07.812724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:07.953580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1eff7-37da-4eaf-86c1-bd3d4840cec2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:07.971536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:07.997692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:07.998308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:07.998587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:07.998885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.072102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:08.073480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:08.178265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 145/292] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:08.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:08.312027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa9bf-b0f2-47e5-ba1d-cc8df4d6ccca None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:08.354011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:08.354665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.354956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.355231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.356366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:08.357808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400ce4d-6857-4dee-b35c-849ba3d6c30c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:08.359774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 148/293] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:08 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:08.365389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:08.380646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:08.477673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78edb6-488f-4252-b8bf-2e180fbd1608 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:08.514293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c2ca4-af9f-4fe7-aa87-15d57acf1dd6 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.514293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c2ca4-af9f-4fe7-aa87-15d57acf1dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.514293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c2ca4-af9f-4fe7-aa87-15d57acf1dd6 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.514293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 149/294] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:08.516886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:08.516886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.516886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.516886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.517551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=21369df04f1e4551b886c336b5b59e14) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:08.517863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:08.518661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0a10-22d8-45ca-a735-cdee4e7d272b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.518986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0a10-22d8-45ca-a735-cdee4e7d272b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.519267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0a10-22d8-45ca-a735-cdee4e7d272b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.519380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e060c0-730a-4e88-9247-b7da35f0bbed None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:08.534945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 146/295] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:08 2022] GET /identity/v3/role_assignments?user.id=21369df04f1e4551b886c336b5b59e14&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:08.553285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-825a0a10-22d8-45ca-a735-cdee4e7d272b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:08.554510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-825a0a10-22d8-45ca-a735-cdee4e7d272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:08.578710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:08.664503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 150/296] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:08.855378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80a8b-ac21-4fed-9935-fde6caf3a929 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.855707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80a8b-ac21-4fed-9935-fde6caf3a929 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.856001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80a8b-ac21-4fed-9935-fde6caf3a929 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.857624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 147/297] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:08.865652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:08.865938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:08.866217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:08.891078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:08.892253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:08.979386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 151/298] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:08.991691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de80a8b-ac21-4fed-9935-fde6caf3a929 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.076502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.113044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de80a8b-ac21-4fed-9935-fde6caf3a929 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.151310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.151947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.152230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.152506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.165251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5b671866-61ac-420b-83a0-ca68bae8a524 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:09.170303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:09.171211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:09.172006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 148/299] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:09.177979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.221513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01d7-56ff-4a95-b13c-6f8143833073 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.222459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01d7-56ff-4a95-b13c-6f8143833073 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.222952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e01d7-56ff-4a95-b13c-6f8143833073 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.225201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 149/300] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:09.230874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.231569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.232053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.256715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:09.258289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:09.267252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.289893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a4fc9-f473-4f9a-b0bc-d4382609c29b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.329557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.330212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.330493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.330846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.331521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:09.331953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:09.333925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:09.347558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 152/301] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:09.347558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.351232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 150/302] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:09.351917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.439572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.455328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090c64f-65e8-4528-9fed-126fd88ef86f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.491807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.492744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.493213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.493654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.494192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce52a-590c-426f-a7e0-7daa249ae4e1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.495173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:09.496780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:09.498500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 153/303] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:09.503893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.516992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.541671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.542586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.542899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.543179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] PATH_INFO: `/v3/users/01953cae74bc4878ac496c050a6a409f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.560812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] RBAC: Authorizing `identity:get_user(user_id=01953cae74bc4878ac496c050a6a409f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:09.562553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:09.565298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 151/304] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/users/01953cae74bc4878ac496c050a6a409f => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:09.571656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.627697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800567e-15dc-4b0a-af02-9edd430b100e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.663017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.663638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.663925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.664203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] PATH_INFO: `/v3/groups` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.664833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:09.666269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e995b4-6ed5-452a-b807-b3b64c13da8b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:09.681831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 154/305] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 15:24:09 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:24:09.682762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc159-c7de-40eb-bb7e-114726fb2e47 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.683050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509c74-cdeb-4564-b08e-7f0aab8600c6 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.683338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509c74-cdeb-4564-b08e-7f0aab8600c6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.683622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509c74-cdeb-4564-b08e-7f0aab8600c6 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.684906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 155/306] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:09.687601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc509c74-cdeb-4564-b08e-7f0aab8600c6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:09.708041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.727565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.728254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.728470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.728727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.729491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, user.id=01953cae74bc4878ac496c050a6a409f, scope.system=all) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:09.729803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:09.731352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380a797-bf55-404c-af9c-2b04f6575d96 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:09.746695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 152/307] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/role_assignments?user.id=01953cae74bc4878ac496c050a6a409f&scope.system=all&role.id=8706fc9e7b654363a1548456569dc351 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:09.746695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.746695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.746695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.764707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:09.765974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:09.792930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:09.852120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc509c74-cdeb-4564-b08e-7f0aab8600c6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:09.868710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 153/308] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:09.897400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:09.898149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:09.898503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:09.898881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:09.910333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:09.915686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:09.916590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:09.917374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 156/309] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:09.924416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.030413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7db834-54e7-4701-8a0b-204d0d178293 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.049879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:10.070762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.071377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.071661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.071939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.072611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:10.072918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.074310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:10.084213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 154/310] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:10.089505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.170098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.209320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a09530-5bf8-48a1-9fa1-ddf7a476b95b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.245478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:10.254069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.255359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.255681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.256041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] PATH_INFO: `/v3/users/031d4b528373469fa9e14131bff16952` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.257930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] RBAC: Authorizing `identity:get_user(user_id=031d4b528373469fa9e14131bff16952)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.259780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:10.261956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 157/311] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:10 2022] GET /identity/v3/users/031d4b528373469fa9e14131bff16952 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:10.267425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.299471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ebb99-d9f0-4f4f-88b8-f0f72fc6973a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.337146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.337924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.338325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.338755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.349804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:10.355359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:10.356388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:10.357370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:10.362738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 155/312] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:10.365751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.376038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d413f5-4111-4c16-a229-7a9ebfa0cc00 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.404064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:10.414282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.414977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.415261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.415544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] PATH_INFO: `/v3/system/users/031d4b528373469fa9e14131bff16952/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.416963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=031d4b528373469fa9e14131bff16952, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.418366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:10.431705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 158/313] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 15:24:10 2022] PUT /identity/v3/system/users/031d4b528373469fa9e14131bff16952/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:24:10.484297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d45af-556b-4d16-87e9-7734e758ef65 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.523609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.524621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.525120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.525593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.526472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:10.527018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.528655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:10.539498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 156/314] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:10.545750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.623188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:10.667138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c09748-d0b9-4f5f-b3be-201211d9b0da None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.707179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.708329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.708818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.709291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] PATH_INFO: `/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.710678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f5a10ef3ce48b8a1664802e700b3a6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.712360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:10.714246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 159/315] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:10 2022] GET /identity/v3/users/f3f5a10ef3ce48b8a1664802e700b3a6 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:10.720418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:10.827220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:10.831925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5180-4603-429f-91d1-4c93cdecb8df None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:10.870561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:10.871476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:10.871917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:10.872350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:10.945561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:10.947286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:11.051229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:11.058972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 157/316] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:24:11.064685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:11.181983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ee97-6f69-4592-a6a0-d027723d3699 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:11.212391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:11.219758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:11.220636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.221094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.221546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.222894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:11.225079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dcd78-42bd-41c0-8801-c23e9b13aca6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:11.227311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 160/317] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:11 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:11.233303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:11.338319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f69a9-5d99-41b0-9333-62b7ee29192d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:11.374632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:11.375270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.375554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.375833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.376574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6, user.id=f3f5a10ef3ce48b8a1664802e700b3a6) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:11.376885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:11.378338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e23f7-1c37-4a96-a51c-da4e5ef2f6c9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:11.385293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 158/318] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:11 2022] GET /identity/v3/role_assignments?user.id=f3f5a10ef3ce48b8a1664802e700b3a6&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=8706fc9e7b654363a1548456569dc351 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:11.718818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef145ad-548e-44fe-b523-cc3d2b7c4abf None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.719591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef145ad-548e-44fe-b523-cc3d2b7c4abf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.719668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef145ad-548e-44fe-b523-cc3d2b7c4abf None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.721081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 161/319] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:11.727234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800655-660a-46b2-81ca-e2828ce239da None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.727600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800655-660a-46b2-81ca-e2828ce239da None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.727949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800655-660a-46b2-81ca-e2828ce239da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.747207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-41800655-660a-46b2-81ca-e2828ce239da None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:11.748399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-41800655-660a-46b2-81ca-e2828ce239da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:11.789631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608512-b633-4d39-899c-8b77efa262dd None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.789955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608512-b633-4d39-899c-8b77efa262dd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.790197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608512-b633-4d39-899c-8b77efa262dd None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.793551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 162/320] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:11.796645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:11.796934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:11.797209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:11.813968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:11.815355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:11.816567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:11.828805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 159/321] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:11.888081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 163/322] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:12.103712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800655-660a-46b2-81ca-e2828ce239da None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.166346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.214877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800655-660a-46b2-81ca-e2828ce239da None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.252167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.252800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.253087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.253360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.254463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:12.255980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:12.262183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 160/323] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:12.263015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.269774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391f7b1-ca09-4295-bcab-9d942e6f5415 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.300145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:12.309086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.309896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.310313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.310751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.328335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:12.328858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:12.333845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:12.334976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:12.336253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 164/324] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:12.340585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.378482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c53e0-8996-476f-beda-dc426fd0e4a2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.415919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.416531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.416810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.417080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] PATH_INFO: `/v3/groups` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.417712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:12.419592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f03d84-6609-4a0f-8623-29b733df779c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:12.442971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 161/325] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 15:24:12 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:12.442971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6f7e-4250-4f47-aa38-0ae462c31e52 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.442971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6f7e-4250-4f47-aa38-0ae462c31e52 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.442971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc6f7e-4250-4f47-aa38-0ae462c31e52 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.442971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 162/326] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:12.447440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be43f2-2669-4ea7-a440-da48f25c6137 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.452400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.452698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.452941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.472644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:12.473931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:12.488369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.489064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.489960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.490236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.490969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:12.491278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:12.492713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:12.501792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 165/327] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:12.508103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.552247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 163/328] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:12.563203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:12.622509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd5f1-5a38-4bc5-a5c3-9df0d6a91b88 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.658298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.658921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.659198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.659469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] PATH_INFO: `/v3/users/01953cae74bc4878ac496c050a6a409f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.660597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] RBAC: Authorizing `identity:get_user(user_id=01953cae74bc4878ac496c050a6a409f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:12.662798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:12.664554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 166/329] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:12 2022] GET /identity/v3/users/01953cae74bc4878ac496c050a6a409f => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:12.670009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.729992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:12.783405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a43b7-77dd-40fa-a77f-f7f6cff73ed7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.821577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.822239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.822518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.822894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] PATH_INFO: `/v3/system/users/01953cae74bc4878ac496c050a6a409f/roles/8706fc9e7b654363a1548456569dc351` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:12.824017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:12.824521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=01953cae74bc4878ac496c050a6a409f, role_id=8706fc9e7b654363a1548456569dc351)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:12.825965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:12.837901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 164/330] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 15:24:12 2022] PUT /identity/v3/system/users/01953cae74bc4878ac496c050a6a409f/roles/8706fc9e7b654363a1548456569dc351 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:12.950880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab1311-a9fe-46bb-9911-a3f560a078c0 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:12.971460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:12.993131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:12.994038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:12.994500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:12.994952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:13.007892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:13.013658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:13.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:13.015831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 167/331] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:13.021540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:13.147269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ba0f-cf43-4be6-bd86-e847345ce407 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:13.170532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:13.188233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:13.189184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:13.189662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:13.190131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:13.191037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:13.191573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:13.193770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:13.204283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 165/332] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:13.210245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:13.332665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9426c-2cf3-4739-9c05-22827fcb9f96 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:13.374868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:13.375516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:13.375802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:13.376081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] PATH_INFO: `/v3/users/031d4b528373469fa9e14131bff16952` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:13.377235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=031d4b528373469fa9e14131bff16952)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:13.378712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c381b3-ef90-4d19-9268-8bf56e984ca7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:13.381181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 168/333] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:13 2022] GET /identity/v3/users/031d4b528373469fa9e14131bff16952 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:13.387182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:13.523972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f372db-698d-41cf-8ada-6e9be21b3c4d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:13.566535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:13.568114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:13.568570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:13.569010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:13.569907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] RBAC: Adding query filter params (role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, user.id=031d4b528373469fa9e14131bff16952, scope.system=all) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:13.570474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:13.572110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cb98b-918f-4bd5-a9e7-47978c0b601d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:13.588818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 166/334] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 15:24:13 2022] GET /identity/v3/role_assignments?user.id=031d4b528373469fa9e14131bff16952&scope.system=all&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:14.867623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c89ae-7390-449a-9fb2-1e4187ef1ab2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:14.868150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c89ae-7390-449a-9fb2-1e4187ef1ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:14.868226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c89ae-7390-449a-9fb2-1e4187ef1ab2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:14.869798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 169/335] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:14.885811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314710e-77ba-408c-bb40-311fa4214a92 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:14.886713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314710e-77ba-408c-bb40-311fa4214a92 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:14.887200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314710e-77ba-408c-bb40-311fa4214a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:14.915624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d314710e-77ba-408c-bb40-311fa4214a92 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:14.917150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d314710e-77ba-408c-bb40-311fa4214a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:14.946144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:14.989815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 167/336] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:15.133062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8188c2-70ad-408c-922f-b5b6d0820688 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.133674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8188c2-70ad-408c-922f-b5b6d0820688 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.134139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8188c2-70ad-408c-922f-b5b6d0820688 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.135899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 170/337] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:15.137557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a893f1-7c8a-4b1b-93f7-ecf12fa1bf6a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.138060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a893f1-7c8a-4b1b-93f7-ecf12fa1bf6a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.138541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a893f1-7c8a-4b1b-93f7-ecf12fa1bf6a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.140531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 168/338] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:15.146676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.146676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.146676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.152789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.153327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.153791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.167296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:15.168582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:15.182226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:15.183445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:15.251017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:15.280513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 169/339] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 15:24:15.281444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 171/340] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:15.383486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:15.495241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:15.504618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bd422-09df-443b-88b3-47471f3abadb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:15.542098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:15.542823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.543111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.543397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.554328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:15.559268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:15.560145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:15.560919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 170/341] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:15.566135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:15.685353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b3647-affe-407b-8c03-645fdda0c422 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:15.723081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:15.723697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.724508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.724822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.725537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:15.725848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:15.727257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:15.736527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 172/342] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:15.742290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:15.747989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:15.791171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:15.862011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade955e7-e7f6-40f7-b3f5-70a012397176 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:15.863235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4d1a2-039f-433a-8837-2797194cd99a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:15.910468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:15.912376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.912814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.913328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.919411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:15.921306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:15.921590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:15.921872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:15.926571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:15.932552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:15.933461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:24:15.934250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:15.936986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 171/343] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:15.938024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:15.960885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.048546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.050018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198494ee-2684-4c20-9c62-f6d7871122db None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.051778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057b1b0-b7aa-4a23-97ad-1bbacdc6831a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.092246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.093236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.093723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.094190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.106074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.109363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:16.116799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:16.117711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:16.118524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:16.120907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 172/344] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:16.121861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.173082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 173/345] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:16.174338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0188a7-5bfc-4c5a-a886-183362f9d3a3 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.174624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0188a7-5bfc-4c5a-a886-183362f9d3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.178990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0188a7-5bfc-4c5a-a886-183362f9d3a3 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.180560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 174/346] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:16.181290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0188a7-5bfc-4c5a-a886-183362f9d3a3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.217899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.254349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72157f8-90ac-4ce9-812a-d770406bf039 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.304070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.305144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.305634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.306197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.307159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:16.307726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.309564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dd481-bc31-405e-ae18-ebb3d2e6c1da None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.315457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0188a7-5bfc-4c5a-a886-183362f9d3a3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.332669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 173/347] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:16.334882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.335477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.335972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.354383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.355047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.355315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.355587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.356259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:16.356558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.357803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:16.357922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.359517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:16.364541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.368490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 175/348] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:16.374623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.440484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 174/349] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:16.441654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.511107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66109cc-0eea-4114-ac06-e04dcfec9bb6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.515988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.556816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54b0e-bdac-4b06-bf41-431f383d908a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.590539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.591199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.591450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.591724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.592992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.594875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.596682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.596897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 176/350] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:16.597644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.597995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.598483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.598971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] PATH_INFO: `/v3/users/01953cae74bc4878ac496c050a6a409f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.604143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] RBAC: Authorizing `identity:get_user(user_id=01953cae74bc4878ac496c050a6a409f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.605862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.608579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 175/351] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/users/01953cae74bc4878ac496c050a6a409f => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:16.609574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.714448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.779987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7005-e228-4231-835f-9d3175baf6a9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.807372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c3bc-c212-4f6e-807f-3d8efa75afb5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:16.839520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.840984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.841599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.842074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.875127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:16.875775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:16.876053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:16.876378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:16.879083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:16.887659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.889123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:16.891176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 176/352] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:16.892955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:16.932832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:16.935833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.015995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5a78d-8f0d-45eb-bebd-3d472fb1781d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.032426 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.066841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.067558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.067848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.068298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.069040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] RBAC: Adding query filter params (role.id=8706fc9e7b654363a1548456569dc351, user.id=01953cae74bc4878ac496c050a6a409f, scope.system=all) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:17.069351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.070863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.078070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 177/353] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:17.078764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.093893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 177/354] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 15:24:16 2022] GET /identity/v3/role_assignments?user.id=01953cae74bc4878ac496c050a6a409f&scope.system=all&role.id=8706fc9e7b654363a1548456569dc351 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.095382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.185644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.224671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8f48c-b8a9-412c-b595-46d500dfe330 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.247801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee8d9-5284-4c84-99de-ffdc0ccd4fa7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.263313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.263964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.264245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.264510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.278596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ceb6ce0f-d3fc-47d0-8c9a-df47eb455748 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:17.283529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:17.284395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:17.285184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 178/355] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:17.285855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9024-8fb3-4f30-804f-c09d17bbc886 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.286405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9024-8fb3-4f30-804f-c09d17bbc886 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.286405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9024-8fb3-4f30-804f-c09d17bbc886 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.287815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 179/356] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:17.289132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9024-8fb3-4f30-804f-c09d17bbc886 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.289240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.289240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.289498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.289687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.290519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=e0dfcf0f119a48eebacae56532f7a756) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:17.290863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.292709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.299399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 178/357] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.300736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.326624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.424348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9024-8fb3-4f30-804f-c09d17bbc886 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.424541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3ad07-7824-44c4-b8e8-abe73539e61d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.456848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.466545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.467225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.467481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f0725-2af6-463e-9490-97726927abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.467596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.467766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.468105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f0725-2af6-463e-9490-97726927abfb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.468489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f0725-2af6-463e-9490-97726927abfb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.468777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f0725-2af6-463e-9490-97726927abfb None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.469352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.469449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f0725-2af6-463e-9490-97726927abfb None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:17.469768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f0725-2af6-463e-9490-97726927abfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.471142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a35e14-62e0-4e4d-a968-96be5816bd3a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.472042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f0725-2af6-463e-9490-97726927abfb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.472825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 180/358] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.474086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.474478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.474797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.483692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 179/359] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.484524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f0725-2af6-463e-9490-97726927abfb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.494788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:17.495406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:17.574161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 181/360] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:17.574962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.604441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f0725-2af6-463e-9490-97726927abfb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.606234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.648600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.649385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.649673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.649954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.651116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.652684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaff81-70ca-48c7-acb1-0dad91cdda21 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.654544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 180/361] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.656132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b600-0506-4dfa-8768-028dc2e87691 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.656540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b600-0506-4dfa-8768-028dc2e87691 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.656817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b600-0506-4dfa-8768-028dc2e87691 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.658460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 181/362] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:17.663667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.663947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.664220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.685561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:17.686875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:17.690770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd696c-eea7-42cd-8232-349e6e69e793 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.734949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.735565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.736215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.736496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.740952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.772262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 182/363] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:17.773365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:17.882964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.884358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.901800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861581f-e83f-4dd5-9bd8-d2fdd79cdd59 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:17.948541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:17.949038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:17.949411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:17.949693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:17.949970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:17.950808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=e0dfcf0f119a48eebacae56532f7a756) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:17.951125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:17.952573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:17.959683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 183/364] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:17.967694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.052706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.093180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 182/365] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:18.099371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.116706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b327a-0ac5-4154-b403-237af4a84a19 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.163286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.164025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.164303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.164580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.178913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:18.184345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:18.185317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:18.186135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 184/366] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:18.186829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.215954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf149e-26a6-41e8-92cd-ea86f47bbce3 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.253603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.266388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.267052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.267327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.267602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.269319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.271691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.274265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 183/367] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:18.274876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.354584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c77ad-e37a-4b8d-9340-d23eac6dba9c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.359909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.402683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.403342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.403624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.404227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.420149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:18.425830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa2878-b5e4-4fc3-b10d-5bc205cfcf71 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:18.427036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:18.428628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:18.429421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:18.430028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 185/368] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:18.434743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.466937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.467629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.468152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.468684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.470626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:18.471291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.472957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.482959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 184/369] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:18.488254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.560283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106bb0-dd52-448e-9436-e6c02211824b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.584813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.611357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233427c1-c184-48d8-84da-991284d4deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.612245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233427c1-c184-48d8-84da-991284d4deab None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.612695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233427c1-c184-48d8-84da-991284d4deab None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.613161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233427c1-c184-48d8-84da-991284d4deab None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.614668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233427c1-c184-48d8-84da-991284d4deab None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.617923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233427c1-c184-48d8-84da-991284d4deab None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.620597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 186/370] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:18.622000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233427c1-c184-48d8-84da-991284d4deab None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.657593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed482a5-eaed-4b06-993b-209afd731fab None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.683098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.699839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.700601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.702173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.702551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.703405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:18.703787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.705260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.718794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 185/371] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:18.719854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.751050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233427c1-c184-48d8-84da-991284d4deab None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.787919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.788810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.789251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.789658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.832882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766a8e-0fa1-48b5-9767-c5b6f841a946 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:18.852317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:18.862872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.864521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.874152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:18.874867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:18.875253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:18.875603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:18.876404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:18.876783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:18.878284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:18.887865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 186/372] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:18.888892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:18.911113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.062411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e7a9b-235b-4f57-b2d8-94736569dc2f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.086195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 187/373] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:19.097096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.116015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.117127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.118041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.119326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.196194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.198810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.201033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.272130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17440207-e17c-42c8-9d02-4565e08a646f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.318779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.323685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 187/374] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:19.330829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.340554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.341739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.342233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.342698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.346762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.348423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.352713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 188/375] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:19.376165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.486581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.515613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d398-b0f2-4793-a8f0-cf3d3fe6c761 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.559505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.560156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.560439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.560718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] PATH_INFO: `/v3/projects/5e88980106c84e4fadc67759de8e9449` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.562345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] RBAC: Authorizing `identity:get_project(project_id=5e88980106c84e4fadc67759de8e9449)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.563825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.567789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cb3a2-eb69-4160-9276-853a485395a7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.570067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 188/376] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/projects/5e88980106c84e4fadc67759de8e9449 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:19.571750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.642895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.643542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.643826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.644102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.645243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.646761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.648502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 189/377] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:19.654371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.722093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0ee58-956e-4280-bb22-9236d7f803d9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.762982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.763859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.764286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.764705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] PATH_INFO: `/v3/groups/9ae477d3f4b9401fb3ac893714326558` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.764993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.773004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] RBAC: Authorizing `identity:get_group(group_id=9ae477d3f4b9401fb3ac893714326558)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.774696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78eafbc-3c83-4bcb-a0a7-cefe76a3a1ba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.776507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 189/378] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/groups/9ae477d3f4b9401fb3ac893714326558 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:19.777960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d28d-9ffa-4712-8a4c-b623cdcaa312 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.778449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d28d-9ffa-4712-8a4c-b623cdcaa312 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.779674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d28d-9ffa-4712-8a4c-b623cdcaa312 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.781080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 190/379] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:19.785909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3d28d-9ffa-4712-8a4c-b623cdcaa312 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:19.805359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:19.811797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035321e3-bd1f-4888-99c4-bfd0e7ad09a6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.859946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.861026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.861759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.862262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.863265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:19.863804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.866296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa554c63-5d9e-4b4e-9576-3173031c7cde None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.875868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 190/380] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:19.877691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.878312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.883087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.919758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:19.921329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:19.929335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3d28d-9ffa-4712-8a4c-b623cdcaa312 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:19.973163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:19.973780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:19.974064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:19.974408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:19.975280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] RBAC: Adding query filter params (group.id=9ae477d3f4b9401fb3ac893714326558, role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=5e88980106c84e4fadc67759de8e9449) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:19.975597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:19.977270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb0d2-128b-41ef-9056-51e661287f2e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:19.984691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 191/381] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:19 2022] GET /identity/v3/role_assignments?group.id=9ae477d3f4b9401fb3ac893714326558&scope.project.id=5e88980106c84e4fadc67759de8e9449&role.id=826e2155943c48ab9c01f294b1734d83 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:20.037203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 191/382] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:20.398989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7d23-adbb-4c91-aa42-01f30cd55bf6 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:20.399522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7d23-adbb-4c91-aa42-01f30cd55bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:20.399522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7d23-adbb-4c91-aa42-01f30cd55bf6 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:20.401130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 192/383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:20.405907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:20.421674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:20.422300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:20.422824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:20.460419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:20.462001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:20.491170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:20.528035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72b7c9-992f-4032-ac21-cfaa4689d700 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:20.575977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:20.576953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:20.578567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:20.579637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:20.600429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 193/384] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:20.602590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:20.610748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:20.611644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:20.612446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 192/385] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:20.616305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:20.632405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:20.751792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54a573-0789-4f07-a6b0-8d41df048726 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:20.794065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:20.795063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:20.795565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:20.796062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:20.796932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:20.797436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:20.799027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:20.814597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 194/386] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:20.820200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:20.961671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493d401-df5c-4629-8596-41b2f3849855 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:20.995038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:21.012923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.013572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.013857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.014134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.015318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:21.016786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:21.018587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 193/387] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:20 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:21.024126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.108043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:21.154228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa819b-6853-4168-bd31-198f4020e8a1 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:21.208238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.208891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.209784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.210065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.211276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:21.214795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:21.216415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 195/388] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:21.222675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.236943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.369398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31e44-b42e-4c44-b060-9e6e260a86a3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:21.406105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.406854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.407096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.407371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/groups/e59108a450cc4db7b32f6d92665c909c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.409293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, group_id=e59108a450cc4db7b32f6d92665c909c, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:21.410776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54cfcf-94b1-4be9-982b-aad0acfc08a6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:21.425580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375ae79-dee4-4034-ad70-3ba89dfbd27d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:21.432625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 194/389] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 15:24:21 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/groups/e59108a450cc4db7b32f6d92665c909c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:21.434291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426fc44-741d-4798-b50c-a723aac43ed5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.434807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426fc44-741d-4798-b50c-a723aac43ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.435307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426fc44-741d-4798-b50c-a723aac43ed5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.436946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 195/390] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:21.444537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.445168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.445588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.465530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:21.467116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:21.469954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.471669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.471669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.471669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.480579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:21.486363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:21.492855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:21.493860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:21.494970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 196/391] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:21.500743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.553501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:21.598713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 196/392] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:21.673082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f211b-b425-4e45-bbe1-a12095b459c8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:21.713519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.714160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.714446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.714759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.715479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:21.715843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:21.717232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:21.731341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 197/393] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:21.736776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.889460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5293e-550c-46dc-9fbc-314d7773bb25 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:21.921543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.944773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:21.945423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:21.945702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:21.945976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:21.947459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:21.948979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:21.952461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 197/394] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:21.956373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:21.990094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.036203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a106d-8a7a-4dac-9dba-36f5bc0da5c9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.047390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.080814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.081466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.081749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.082025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.104592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:22.104797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1203d-e856-46e8-8bc3-34796078108d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:22.109978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:22.110944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:22.111789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 198/395] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:22.117175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.142935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.143844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.144271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.144711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.145997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.148601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.151458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 198/396] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:21 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:22.159644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.261599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e939b83-548a-4243-8406-832e8a48599d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.282546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aae3e8-3361-451e-8637-94ad686cf393 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.284052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.306435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.307350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.307811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.308237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.309066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:22.309524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.311083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896053d-b898-45ee-a462-70e721850e7f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.329204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.329689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 199/397] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:22.333724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37f663-8d86-4ce1-bfb5-cac6a3824791 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.333998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37f663-8d86-4ce1-bfb5-cac6a3824791 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.335409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37f663-8d86-4ce1-bfb5-cac6a3824791 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.338678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 200/398] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:22.339875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37f663-8d86-4ce1-bfb5-cac6a3824791 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.361654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.362722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.363258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.363760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756/groups/e59108a450cc4db7b32f6d92665c909c/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.367174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] RBAC: Authorizing `identity:create_grant(project_id=e0dfcf0f119a48eebacae56532f7a756, group_id=e59108a450cc4db7b32f6d92665c909c, role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.368922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b909d9-ffe1-4e68-9b08-d2809406b05f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.388059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 199/399] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 15:24:22 2022] PUT /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756/groups/e59108a450cc4db7b32f6d92665c909c/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:22.389764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.394518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.395530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.429524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:22.430765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:22.511347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37f663-8d86-4ce1-bfb5-cac6a3824791 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.533883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 200/400] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:22.563223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.563916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.564200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.564592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.565752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.567294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.569369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 201/401] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:22 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:22.575281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.695119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f407d-f477-494b-ab0a-1802a3553f1d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.703052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.734988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.735626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.735908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.736185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.737315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.738785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.741239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 201/402] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:22 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:22.746921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.805618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:22.864561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63944c6-7581-47db-a3c9-473e58d1f5f1 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:22.902737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:22.903462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.903745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.904068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/groups/e59108a450cc4db7b32f6d92665c909c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.905999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, group_id=e59108a450cc4db7b32f6d92665c909c, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:22.907465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa44e3-ef1e-47aa-9dca-95167540fc6c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:22.927470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 202/403] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 15:24:22 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/groups/e59108a450cc4db7b32f6d92665c909c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:22.970771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:22.986513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71c83-cb82-44c5-9abb-1516d8ef1c7a None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:22.986856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71c83-cb82-44c5-9abb-1516d8ef1c7a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:22.987107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71c83-cb82-44c5-9abb-1516d8ef1c7a None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:22.988825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 203/404] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:23.001405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.001750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.001969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.051099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:23.052386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:23.173327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f2eb-4541-434d-8149-b1239ca67d80 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.177094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 204/405] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:23.220898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.221840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.222118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.222388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.235736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:23.241564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:23.242489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:23.243306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:23.246717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 202/406] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:23.250500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.299246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:23.360692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b70d8-09e6-4c42-87ae-b7e14167c105 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.397327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.398152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.398431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.398723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.399402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:23.399707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:23.401074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:23.409913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 205/407] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:23.416360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.456816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:23.535851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438feb0-3794-4d11-8331-9b319f18e1d4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.579169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.579807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.580086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.580355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.585549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:23.587104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:23.589072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 203/408] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:23 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:23.594356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.637488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.727323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5302a8b-001c-4c54-8baf-5cd1daeeee61 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.728914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:23.764989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.765905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.766187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.766459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.767700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:23.769204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:23.774691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 206/409] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:23 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:23.776594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.812370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce236957-c113-469c-ba4d-b5d5844475ae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.814972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:23.878309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.878995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.879379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.879660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.895577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42ab6-2a47-42aa-8c46-e22fe6aee9f7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:23.900378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:23.905400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:23.906509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:23.907400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 204/410] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:23.914304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:23.967813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:23.969658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:23.970119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:23.972381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/groups/e59108a450cc4db7b32f6d92665c909c/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:23.978274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fffc30ec892f44efb2b4397d8e89a1c6, group_id=e59108a450cc4db7b32f6d92665c909c, role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:23.981015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:24.005151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 207/411] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 15:24:23 2022] PUT /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6/groups/e59108a450cc4db7b32f6d92665c909c/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:24:24.068398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf2053-7077-4822-8fa1-865ab1be783b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:24.108022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:24.108926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.109380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.109862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.110752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:24.111200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:24.112886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de3bd6-20be-4eb7-b8b2-1de15b8393e0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:24.123137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 205/412] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:24.129010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:24.144722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303d5a-a08d-4db7-afd2-9504870974b1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.145243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303d5a-a08d-4db7-afd2-9504870974b1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.145697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303d5a-a08d-4db7-afd2-9504870974b1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.147399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 206/413] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:24.153131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.153934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.154607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.187417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:24.191888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:24.203514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:24.223086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:24.286836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 207/414] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:24.293977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76f64-a9fa-40a6-b163-123bb355eef2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:24.333959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:24.335161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.335506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.335839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] PATH_INFO: `/v3/groups/9ae477d3f4b9401fb3ac893714326558` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.337020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] RBAC: Authorizing `identity:get_group(group_id=9ae477d3f4b9401fb3ac893714326558)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:24.338467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:24.340487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 208/415] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:24 2022] GET /identity/v3/groups/9ae477d3f4b9401fb3ac893714326558 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:24.345970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:24.454754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cebd07-8114-4bf4-8fdc-c035fcdc0c09 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:24.492687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:24.494084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.494586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.495091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] PATH_INFO: `/v3/projects/5e88980106c84e4fadc67759de8e9449` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.496926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] RBAC: Authorizing `identity:get_project(project_id=5e88980106c84e4fadc67759de8e9449)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:24.498608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:24.500795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 208/416] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:24 2022] GET /identity/v3/projects/5e88980106c84e4fadc67759de8e9449 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:24.506813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:24.609045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:24.630441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673dcd-f59e-464f-9838-996abe40d26c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:24.636928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:24.651686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:24.710491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:24.711124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.711408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.711684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] PATH_INFO: `/v3/projects/5e88980106c84e4fadc67759de8e9449/groups/9ae477d3f4b9401fb3ac893714326558/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.715406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e88980106c84e4fadc67759de8e9449, group_id=9ae477d3f4b9401fb3ac893714326558, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:24.716855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:24.736441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 209/417] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 15:24:24 2022] PUT /identity/v3/projects/5e88980106c84e4fadc67759de8e9449/groups/9ae477d3f4b9401fb3ac893714326558/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:24.792923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e80d3-feed-4539-9499-2dcb01de122f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:24.844940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:24.845601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.845885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.846161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.860513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b26987b2-e6cc-4e3a-8458-4e8274771399 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:24.865643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:24.867220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:24.868018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 209/418] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:24.874803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:24.992242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ece28-a549-4f2f-bcce-b6b19af4f709 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.992565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ece28-a549-4f2f-bcce-b6b19af4f709 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:24.992793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ece28-a549-4f2f-bcce-b6b19af4f709 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:24.994286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 210/419] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:24.999632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:24.999924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.000202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.002130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52791950-c806-4a89-bcf9-de5b81c879e8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.021560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:25.023999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:25.053118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.053972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.054257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.054535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.055236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] RBAC: Adding query filter params (name=member) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:25.055547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:25.056922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:25.073704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 210/420] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:25.079454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.123372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:25.149774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 211/421] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:25.211995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a0d4-c95b-4e5e-967a-0d1326dc533a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.249551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.250161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.250449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.250753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.321851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:25.323444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:25.346036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:25.431050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 211/422] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:25.436378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.456426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:25.558206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.573571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99a572-0254-4130-9214-d069a4bf71a9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.638185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.638956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.639241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.639522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.640723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:25.642180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:25.644358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 212/423] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:25.651553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.663515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bac537-de4d-4661-9e50-00e87017ffd5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.686961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:25.703928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.704565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.704843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.705114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.715969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:25.720960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:25.722118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:25.723038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 212/424] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:25.728913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.820426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:25.845798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6601a-f65d-4e57-a9e0-0304ebec2e9f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.864951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9136a-d520-4862-9891-1b6083c8711b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:25.895195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.896100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.896773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.896926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.898195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:25.900322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9cae4-dba2-4712-bb38-4dc53d063ef0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:25.906696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 213/425] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:25.906696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c92b-e724-41cf-a900-9bf3211853e4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.906696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c92b-e724-41cf-a900-9bf3211853e4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.906696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c92b-e724-41cf-a900-9bf3211853e4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.906696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 214/426] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:25.911903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3c92b-e724-41cf-a900-9bf3211853e4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:25.913874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:25.914880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.915474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.915945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.916817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:25.917332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:25.918986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb26dfa-f5bf-4d7b-9e0e-43a020d484ac None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:25.929629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 213/427] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:25.931664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:25.932287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:25.932782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:25.953095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:25.954596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:25.994696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.024998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3c92b-e724-41cf-a900-9bf3211853e4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.035390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 214/428] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:26.036676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.088172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.089165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.089697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.090185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.091342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=e0dfcf0f119a48eebacae56532f7a756) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:26.092997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:26.095400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:26.111662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 215/429] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:25 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:26.202721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c29523-8bfb-4f8e-b343-f931992b17a1 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.240050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.241159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.241644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.242218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.300992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.348200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:26.349015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c9aca-1c16-4431-932a-df9d3a3df116 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:26.367038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.392002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.498278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 215/430] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:26.499597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb5fc-e4f9-4353-8775-b34475d83dfa None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.499881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb5fc-e4f9-4353-8775-b34475d83dfa None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.500154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb5fc-e4f9-4353-8775-b34475d83dfa None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.500995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064ec5-f006-43bb-9f4b-8c96aa28b36b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.505528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 216/431] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:26.506596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fb5fc-e4f9-4353-8775-b34475d83dfa None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.546330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.547041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.547321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.547605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.561325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9fd3b635-8bd8-4132-8ffb-d1231071096b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:26.567063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:26.567969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:26.568758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 216/432] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:26.575021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.575021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.575021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.596490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:26.598441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:26.605995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.646266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fb5fc-e4f9-4353-8775-b34475d83dfa None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.685199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 217/433] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:26.687486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.693458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.696302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.697058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.697409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.697755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] PATH_INFO: `/v3/projects/e0dfcf0f119a48eebacae56532f7a756` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.699712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] RBAC: Authorizing `identity:get_project(project_id=e0dfcf0f119a48eebacae56532f7a756)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:26.701077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:26.707050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 217/434] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/projects/e0dfcf0f119a48eebacae56532f7a756 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:26.710518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.809560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e26f2e-59c7-47a9-9961-96684ba9ddc7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.824837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74335792-1ba8-4309-b9e0-e537a9d7abdb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:26.847831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.848586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.848964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.849354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.850208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:26.851279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:26.852755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:26.861904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:26.862126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 218/435] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:26.862538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:26.862850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:26.863126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:26.864511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:26.865939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:26.868155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 218/436] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:26.869032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.876149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:26.890288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:26.958393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.005024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35755f-c78e-4665-bb1f-01d5999f6817 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.007854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c88c8-b7d2-4b1b-bf3e-4d8acf7d068c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.042811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.043042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.043414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.043622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.043729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.043899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.044002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.044170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.044716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=e0dfcf0f119a48eebacae56532f7a756) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:27.044953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.046423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.061405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 219/437] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=e0dfcf0f119a48eebacae56532f7a756&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:27.061405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.140321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.142078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b75dd-6cb4-4991-b27c-3b1891be5211 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.149678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.184269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e45773-8937-4790-ace4-8d682267fa09 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.217072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.230878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.231532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.231805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.232079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.248849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9e078c90-8f14-4c5b-838a-747ef7a7dd30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:27.254342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:27.255275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:27.256083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 220/438] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:27.257451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b0c2-e749-4de6-9164-a71c2aadea62 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.257929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b0c2-e749-4de6-9164-a71c2aadea62 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.257929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0b0c2-e749-4de6-9164-a71c2aadea62 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.259213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 221/439] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:27.263098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b0c2-e749-4de6-9164-a71c2aadea62 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.279353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 219/440] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:27.281162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.281443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.281715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.302362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:27.304186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:27.390832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0b0c2-e749-4de6-9164-a71c2aadea62 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.391819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 220/441] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:27.399644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.433262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.440319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.441002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.441282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.441567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.442233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:27.442541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.445579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.455911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 222/442] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:27.462946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.517551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.526032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3be47-13bd-41a6-8b57-e094b7f0d418 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.564592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.565223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.565335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1ee71-fadd-45c4-84eb-1d90be83b2e5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.565498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.565769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.566923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.568360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.570827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 221/443] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:27.576004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.602921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.603522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.603801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.604075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.668713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.677276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.679293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad0d-c8dd-47d3-9dad-cc8782467013 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.679422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.714388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.715050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.715333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.715605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.716725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.718154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.719750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 222/444] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:27.720585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.751634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.779793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 223/445] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:24:27.780631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.836944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a777f90-7e7d-4a7b-9e18-5663b9ad02cb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.873384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.874873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.875156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.875422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.881159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.888311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:27.889304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ebe67-321e-4b25-bcf3-98e66c448c67 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:27.893098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:27.894156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:27.895028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 223/446] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:27.895690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.924684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:27.925281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.925562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.925837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.926566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=fdb2cb2ca51e4f0eaa2683395176a5bc, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:27.926930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:27.928348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:27.934188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 224/447] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=fdb2cb2ca51e4f0eaa2683395176a5bc => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:27.935090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.996211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.008849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb82d3-ac60-4e50-b592-0d7239c27e49 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.046188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.046913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.047200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.047479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] PATH_INFO: `/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.048758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] RBAC: Authorizing `identity:get_project(project_id=fffc30ec892f44efb2b4397d8e89a1c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.050244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.051132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b3f8c-9229-4d2d-b127-315df8bbb682 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.052110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 224/448] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/projects/fffc30ec892f44efb2b4397d8e89a1c6 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:28.058528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:28.093456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.093985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.095128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.095128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.095289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:28.095503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.097157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.111532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 225/449] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:28.119148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:28.142516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.193487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3a614-72c1-408d-a18b-eb7302e10ecc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.238922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.239562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.239844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.240127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] PATH_INFO: `/v3/groups/e59108a450cc4db7b32f6d92665c909c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.241851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] RBAC: Authorizing `identity:get_group(group_id=e59108a450cc4db7b32f6d92665c909c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.243516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.245139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 225/450] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/groups/e59108a450cc4db7b32f6d92665c909c => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:28.248538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3783e02-014f-483c-89df-e9036a760f77 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.252973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:28.294752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.296735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.298465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.298870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.299786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.388519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.389912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.400905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88be1b2-382a-4f2b-b5a2-c405a7d4fef3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.412336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.444754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.445519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.445864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.446364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.447830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] RBAC: Adding query filter params (group.id=e59108a450cc4db7b32f6d92665c909c, role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:28.448337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.450267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.456984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 226/451] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/role_assignments?group.id=e59108a450cc4db7b32f6d92665c909c&scope.project.id=fffc30ec892f44efb2b4397d8e89a1c6&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:28.530090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 226/452] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:24:28.536413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:28.649825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.661295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07d922-8928-461c-83aa-9729f42b7143 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.703622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.704489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.704976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.705459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] PATH_INFO: `/v3/projects/5e88980106c84e4fadc67759de8e9449` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.707381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] RBAC: Authorizing `identity:get_project(project_id=5e88980106c84e4fadc67759de8e9449)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.709443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.711647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 227/453] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/projects/5e88980106c84e4fadc67759de8e9449 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:28.717714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:28.842093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810bb16-a3bb-4b75-9c8d-12eb5905607e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.875193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:28.890465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.891123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.891400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.891669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] PATH_INFO: `/v3/groups/9ae477d3f4b9401fb3ac893714326558` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.892901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] RBAC: Authorizing `identity:get_group(group_id=9ae477d3f4b9401fb3ac893714326558)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.894323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19304090-619e-459f-86f2-383fb4ebe8bc None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.896403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 227/454] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/groups/9ae477d3f4b9401fb3ac893714326558 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:28.903362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:29.023514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4869cd-e544-4330-a594-cae341fb0d91 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:29.065683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:29.066287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:29.066564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:29.066859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:29.067583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] RBAC: Adding query filter params (group.id=9ae477d3f4b9401fb3ac893714326558, role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=5e88980106c84e4fadc67759de8e9449) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:29.067889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:29.069358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9135c-be94-4349-9443-f08783d010a9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:29.078494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 228/455] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 15:24:28 2022] GET /identity/v3/role_assignments?group.id=9ae477d3f4b9401fb3ac893714326558&scope.project.id=5e88980106c84e4fadc67759de8e9449&role.id=826e2155943c48ab9c01f294b1734d83 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:31.372594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd617f-73c6-49b6-95a8-eba746df04df None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:31.373117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd617f-73c6-49b6-95a8-eba746df04df None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:31.373193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd617f-73c6-49b6-95a8-eba746df04df None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:31.374785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 228/456] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:31.379795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac450d-6d53-49c9-bfca-ac37fe358317 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:31.380189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac450d-6d53-49c9-bfca-ac37fe358317 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:31.380351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac450d-6d53-49c9-bfca-ac37fe358317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:31.403717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-20ac450d-6d53-49c9-bfca-ac37fe358317 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:31.404931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-20ac450d-6d53-49c9-bfca-ac37fe358317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:31.428208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:31.493566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 229/457] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:31.600712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c2fc-147a-4fcf-b628-99f5ecc48362 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:31.602086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c2fc-147a-4fcf-b628-99f5ecc48362 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:31.602671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c2fc-147a-4fcf-b628-99f5ecc48362 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:31.604540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 229/458] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:31.609641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:31.609931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:31.610211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:31.639612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:31.640784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:31.732344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 230/459] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:31.841239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5c2fc-147a-4fcf-b628-99f5ecc48362 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:31.978375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5c2fc-147a-4fcf-b628-99f5ecc48362 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.015110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.016085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.016569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.017061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.028465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.030240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:32.045701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:32.046679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:32.048149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:32.048149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.048149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:32.050261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 230/460] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:32.056664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.072210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.119012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.178977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2608-24ba-4761-b40b-889ef5ff79c4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.206100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.219987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.220848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.221289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.221717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.230398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebeb33-fb9e-432b-99c9-87c02ccd3306 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.236229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.237679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a66f1b5e-b351-43e8-b04a-e8993b3e2d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:32.253680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:32.254612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:32.255925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:32.255925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.255925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:32.259920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 231/461] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:32.261260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1831-5710-4119-b096-8eafe22688bb None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.261545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1831-5710-4119-b096-8eafe22688bb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.261818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1831-5710-4119-b096-8eafe22688bb None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.263562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 232/462] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:32.264459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1831-5710-4119-b096-8eafe22688bb None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.277738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.279107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.279873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.280359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.281485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:32.282502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.284668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea2a59-63e5-4d3c-b0c7-9dde6a91825e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.302400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 231/463] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:32.304072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.305270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.305811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.332125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:32.334159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:32.388560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.401254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1831-5710-4119-b096-8eafe22688bb None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.427237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 232/464] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:32.428670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.440463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.441300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.441647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.441987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.442785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:32.443160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.444779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.458351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 233/465] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:32.465017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.571789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.594685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968dee1-e218-46e1-b56a-0afe83044f25 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.611697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd8762-c1c7-4a4e-a441-d9a308d8e5cc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.631243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.632025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.632302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.632601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.633242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.634745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be06c-4911-44b7-995e-04e1cc7d0a90 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.650172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.651068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.651411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.651821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.652588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.654203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677480d-8a41-4461-b278-6438fa535f37 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.664835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 234/466] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:32 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:24:32.666549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7fed-4e7d-481c-bea0-0d4c27389f28 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.667045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7fed-4e7d-481c-bea0-0d4c27389f28 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.667495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7fed-4e7d-481c-bea0-0d4c27389f28 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.675747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 235/467] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:32.683015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28708-b592-45a9-9130-db700256b4c0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.683616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28708-b592-45a9-9130-db700256b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.684113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf28708-b592-45a9-9130-db700256b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.684216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 233/468] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:32 2022] POST /identity/v3/users => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:32.704878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-adf28708-b592-45a9-9130-db700256b4c0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:32.706538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-adf28708-b592-45a9-9130-db700256b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:32.709107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.743622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d8a9-04f1-4103-8cdf-c7ff6a4a2eef None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.743921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d8a9-04f1-4103-8cdf-c7ff6a4a2eef None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.744343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d8a9-04f1-4103-8cdf-c7ff6a4a2eef None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.746982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 234/469] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:32.751018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d8a9-04f1-4103-8cdf-c7ff6a4a2eef None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:32.794002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 236/470] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:32.795763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.796352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.796795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.818365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:32.820389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:32.908375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 237/471] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:32.918522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d8a9-04f1-4103-8cdf-c7ff6a4a2eef None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:32.948840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:32.956881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:32.957596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:32.957951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:32.958301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:32.972070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:32.973914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:32.992593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:32.993568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:32.994446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:32.994446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:32.994446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:32.994446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 235/472] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:32.999515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.081671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.131337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22137b-b0f2-47a0-b2c9-50ed49ac6b97 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.162043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.178830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.179478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.179885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.180372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.181051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:33.181359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.184624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.208532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 238/473] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:33.211432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.296571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b537a-6bc6-4b13-9048-d55267b58434 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.331786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557711-e58d-4cb1-b57b-d5175d3ce658 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.352274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.353063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.353463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.353842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.360942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.368135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.370716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.370853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.371522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.371873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.372219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.372990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.374422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.381960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.sql.core [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:24:33.381960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 15:24:33.381960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [parameters: {'id': '805bca84b58a4dafa802b9e78c6212b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 15:24:33.381960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81446) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:33.388647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:33.389537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:33.390398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:33.390398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:33.390398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:33.392523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 236/474] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:33.393526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result.read() Aug 24 15:24:33.407199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:24:33.408463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application yield resource Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise self.value Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application session.commit() Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:24:33.409388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:24:33.410259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:24:33.415442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result.read() Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:24:33.416291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [parameters: {'id': '805bca84b58a4dafa802b9e78c6212b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:33.417098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 15:24:33.417905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.418829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 239/475] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 15:24:33 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:33.418829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.444880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.539146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e85dd-08e1-42c5-8fe6-f3b7d436ecb3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.542842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68c5bc-81ca-4180-b616-264d9b0f80ff None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.576291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.577234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.577730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.578220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.579115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.580742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.587434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.588054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.588337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.588612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.589267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:33.589577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.591343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.606852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 240/476] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:33.610869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.612366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 237/477] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:33 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:33.614824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.672231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.726970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.730050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53549-d4e1-49d5-b782-2af68a504bc6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.742829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8173f-af42-4e15-a2f1-4cb761669dc2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.770545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.771212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.771484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.771752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.781827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.782840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.783321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.783809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.784676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.784780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:33.786424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5032d-9bc9-4c1d-9280-4b7e3a844bfb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:33.790360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:33.791284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:33.792495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 241/478] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:24:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:33.797452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:33.827452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 238/479] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:33 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:33.912138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb4bbc-e16d-4a30-899f-fe8c78c0a1c3 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:33.928220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:33.952626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:33.953256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:33.953539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:33.953817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:33.954505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:33.954841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:33.956249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab4a05-93f4-421e-af2e-2efbb08e5c47 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:33.966132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 242/480] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:35.122116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dff1e-24a0-45ba-b459-00593412d528 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:35.123161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dff1e-24a0-45ba-b459-00593412d528 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:35.123684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dff1e-24a0-45ba-b459-00593412d528 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:35.125229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 239/481] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:35.132648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:35.133687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:35.133869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:35.149217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ee8be-9c33-4fef-9cbf-267673dc01f6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:35.149561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ee8be-9c33-4fef-9cbf-267673dc01f6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:35.149782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ee8be-9c33-4fef-9cbf-267673dc01f6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:35.153037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 240/482] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:35.156670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:35.156956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:35.157234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:35.161639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:35.162881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:35.194405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:35.195689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:35.326671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 243/483] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:35.328000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 241/484] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:35.644793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:35.726466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:35.786098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828146-d256-411c-8a66-24d1e7160f61 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:35.789390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:35.831947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:35.833107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:35.833538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:35.833955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:35.847341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:35.854013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:35.855093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:35.855986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:35.862773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 244/485] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:35.864082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:35.897821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:35.952646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284cd80-3c7d-426a-905e-3dbf05332056 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.015060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967171-b117-49fa-952e-5965c5beb14e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.027436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.028104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.028394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.028675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.040412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:36.045229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:36.046165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:36.047657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 242/486] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:36.052027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.075177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.075805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.076095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.076374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.077068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:36.077374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.080028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.096778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 245/487] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:36.103507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.158016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc95e63-e22b-4b44-bb0e-7c7e310a5c27 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.159017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.195021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.196157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.196442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.196724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.197398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:36.197702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.199117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.207737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 243/488] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:36.211350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.213377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.267431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed008c2-f25b-40f1-a7a5-4b64d6b8f90c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.307040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.307668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.307945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.308328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.327270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.328715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.335120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47ea14-2540-4d34-9403-4c6edc9c3396 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-4988e8c4-def7-4871-bdd9-31781762691d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:36.343885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:36.344800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 246/489] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ffb83-1eec-41b1-8581-7cc7858967ea None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ffb83-1eec-41b1-8581-7cc7858967ea None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ffb83-1eec-41b1-8581-7cc7858967ea None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 247/490] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:36.353022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30f93-330f-4370-80dd-322ba1ede5e0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.353540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30f93-330f-4370-80dd-322ba1ede5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.353540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db30f93-330f-4370-80dd-322ba1ede5e0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.353540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 248/491] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:36.353540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db30f93-330f-4370-80dd-322ba1ede5e0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.383874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.384837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.385316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.385809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.396543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.398503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-71f3bd1d-9b60-450a-b0f0-0845263e7e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:36.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:36.415073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:36.415919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:36.415919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:36.415919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:36.420336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 244/492] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:36.421737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c146-3b96-4902-bd26-3dfc7f3c6ba9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.421968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c146-3b96-4902-bd26-3dfc7f3c6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.422247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463c146-3b96-4902-bd26-3dfc7f3c6ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.438624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.440922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8463c146-3b96-4902-bd26-3dfc7f3c6ba9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:36.442204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8463c146-3b96-4902-bd26-3dfc7f3c6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:36.465998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db30f93-330f-4370-80dd-322ba1ede5e0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.488550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.502389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.503028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.503316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.503599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.504247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:36.504556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.505933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.516133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 245/493] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:36.517249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.517534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.517805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.526333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 249/494] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:36.527200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.544201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:36.546314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:36.622266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 246/495] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:36.623145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.641467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e86-61a9-49de-beef-abe9944837f4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.691107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.692025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.692482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.693083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.693925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:36.694900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.697635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312af93-a98b-44ec-b742-e0e0c8e49676 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.706891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.717671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 250/496] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:36.719254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7b435-82bb-431c-a04a-1540e46a5b23 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.719702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7b435-82bb-431c-a04a-1540e46a5b23 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.720136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7b435-82bb-431c-a04a-1540e46a5b23 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.723248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 251/497] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:36.724459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7b435-82bb-431c-a04a-1540e46a5b23 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.739865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a47ed9-844d-429c-969f-c62ce1ae0cdf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.749865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:36.778801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.779321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.779676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.780060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.788786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.790367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-0fb87cce-ae37-4413-b952-e2a5f53be0ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:36.803674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:36.805212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:36.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:36.809581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 247/498] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:36.810926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.811215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.811498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.829945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:36.831254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:36.856198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7b435-82bb-431c-a04a-1540e46a5b23 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:36.893562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:36.894178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:36.894461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:36.894765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:36.904552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:36.906001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:36.906860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 248/499] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:36.907699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:36.919071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:36.920190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:36.921247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 252/500] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:36.922825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:36.988918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.020880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.026007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9b02-2a7f-4433-bb0c-33c847feaf65 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.037706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4713a-2dd6-4475-8524-3628cb1de12b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.062276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.062946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.063223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.063500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.073590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:37.076929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.077692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.078116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:37.078328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:37.079437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:37.080258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 249/501] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:37.080937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.081306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.082079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:37.082392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.083873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.125523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 253/502] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:24:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:37.126474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.193456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcbfc7-30fd-4b68-8ed2-e04375e9b70e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.228590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.229301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.229584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.229856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.231155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291e854-ecda-4240-8c38-ee9b2bc6abea None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.240025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:37.244744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:37.245621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 250/503] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:37.247122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.249821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.265105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.269825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.270431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.270741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.271021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.271760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:37.272069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.273508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.292060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 254/504] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:37.292869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.349158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb566c-1cfd-4bab-92fb-1d97222ce4e5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.393970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.394574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.394888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.395183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.395828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:37.396131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.397501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.397908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1877f70-4abb-4e4e-a6a1-16dcaa760786 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.405633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 251/505] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:37.406821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.434729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.435339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.435624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.435898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.481488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.490474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.506765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.508753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.510844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85681f-803b-4b94-9e44-fd3749577f5b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.544937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.545554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.545831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.546101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.556652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.558412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:37.572711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:37.573789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:37.574763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:37.574763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:37.574763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:37.574763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 252/506] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:37.574763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.610524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 255/507] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:37.611369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.675952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5aafb-4014-418c-9373-f2d29ee42013 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.684076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.705590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.709726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6b2c5-012b-4f33-a46c-b1ff7a9c63ce None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.711240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.712128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.713210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.713492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.714166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:37.714475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.715878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.724146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 253/508] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:37.724874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.748567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.749096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.749376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.749650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.816474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.817949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.825063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957cde4-6ba1-408a-be23-8d8c33a07a3c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:37.859044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:37.859802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:37.860084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:37.860354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:37.870859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:37.872292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:37.886258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:37.887383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:37.888349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:37.888349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:37.888349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:37.888349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 254/509] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:37.888349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.896581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.923311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 256/510] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:37.924226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:37.934814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:37.987671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e465b3-1ebd-4768-836d-36d48c0ff925 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.021469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.021977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726be1e-a555-46f0-963f-48a661e642ec None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.022080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.022324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.022597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.023270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:38.023576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.025041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.036590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 255/511] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:38.037377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.061868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.062498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.062816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.063093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.073612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.075098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:38.089342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.090371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:38.091272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:38.091272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.091272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:38.091272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 257/512] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.091545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.119934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.143589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73eb897-0a91-4f0e-ba35-3044f15b3bf0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.150659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.178370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.179028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.179368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.179737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.190249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.191734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.194932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dd6cc-2a2f-4c20-8f32-5132539c93f8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:38.206689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.208021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:38.208896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:38.208896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.208896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:38.208896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 256/513] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.208896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.233722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.234322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.234604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.234919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.235560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:38.235864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.237228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.250915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 258/514] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:38.251700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.313983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3bebf-82ba-4bc4-a92d-3a05e2f273d0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.337027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.355094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1d06-f6e5-4cd2-b9ac-6074ccff1755 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.358217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.358877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.359157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.359427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.371642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.373098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:38.387387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.388319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:38.389190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:38.389190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:38.389190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:38.389543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 257/515] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.390456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.391078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.391816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.392085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.392355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.393143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.394578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.405029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.417015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 259/516] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:38 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:38.417822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.527913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381ff7c-a91f-4ba8-b562-a95bccd347b0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.531332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848164-3dde-4f05-bedc-7386ad3c1225 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.568258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.568790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.569081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.569356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.569552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.570168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:38.570327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.570416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.570657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.570970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.571614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:38.571734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.571928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.573496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.584557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 258/517] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:38.585287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.588195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 260/518] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:38.589150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.592753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.661459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.695233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dbb3-cb7c-4408-a07c-3448d4d88116 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.703460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28324bd-3137-41a8-90ef-3382517146cd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.730908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.731534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.731809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.732078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.739139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.739915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.740527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.740854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.741276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.741689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.743191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:38.743650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.745002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:38.752607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.753542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 259/519] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.754464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.759602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 261/520] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:38.760570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.829484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.871566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ff9f8-9293-4d07-b06b-ecf79518b3f4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.877324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde679c7-9d8a-4a72-a8dc-558afdab9acf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:38.907636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:38.912220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.912220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.912220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.912220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.915714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:38.916378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:38.916657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:38.916925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:38.920444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.921890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.924098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:38.926010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:38.933267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.934165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:38.935087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 262/521] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.935668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:38.942313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:38.943354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 260/522] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:38.944208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.049418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ceb66-144a-450a-be9a-d12632ef1bb7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.059978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b424b9-e20e-4300-8a4c-a2b9369c787c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.074156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.087293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.087818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.088099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.088374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.089109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.089413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.090873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.096726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.097355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.097731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.097912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.106263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.107741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.110327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 263/523] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.115335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:39.119759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:39.120778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 261/524] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:39.121658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.175853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.228266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374514-93b6-4d6b-bff8-95c977cc922e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.229846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c4b40-df5d-4d7f-ac51-fd6a43101ce0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.264214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.264867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.265092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.265368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.266093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.266401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.267849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.268310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.269122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.269754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.270972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.271893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.272398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.274846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.290615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 264/525] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.290615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.293774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 262/526] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.298614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.335931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.409505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3454fff-5b77-4e44-abd3-351076d021e7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.421130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80fca2-1509-4a19-9367-1b5d42f211b2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.435619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.449898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.450518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.450818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.451096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.469010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.470185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.470973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.471464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.472350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.472876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.474928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.487652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 263/527] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.494171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.525012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.526948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.569084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.606130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e95611-87fc-49bf-a79a-dba35fbdda36 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.641413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.641996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 265/528] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:39.642108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.642292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.642530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.647293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.686148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.715109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.716588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.755027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb23b-3f99-422c-b40e-c282c8ac9cae None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.788968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.789572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.790439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.790798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.791536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.791846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.793286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.798959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 266/529] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.799748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.804314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.846017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 264/530] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:39.852889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.926589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b848b-03fa-4491-b1d1-c0f56a761193 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:39.955118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:39.969132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:39.970097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:39.970582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:39.971086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:39.972031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=f978768101cd4e478b10b2f919383518) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:39.972586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:39.974285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:39.981121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 267/531] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/role_assignments?user.id=f978768101cd4e478b10b2f919383518&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:39.982072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:39.992117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e41e9-dd43-48aa-8e4d-4fcd9bd8ebc1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.044450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.045097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.045379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.045651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.057515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:40.069710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.071285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:40.099404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:40.100390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:40.101236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:40.101236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:40.101236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:40.105016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01a966-db24-4436-819b-1923e5082305 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.107062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 265/532] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:40.109815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.181494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.182142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.182537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.182989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.203849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.205388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:40.239434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:40.240415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:40.241290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:40.241290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:40.241290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:40.245296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 268/533] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:40.250729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.320708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e9019-0023-4ef7-9ee6-f667c9cba063 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.371583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:40.392566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.393185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.393470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.393745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.394404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:40.394739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.396116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f353873-28f1-4dda-907c-a4d938c2d2fe None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.416784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 266/534] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:40.418196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a3272-db17-446f-9ef4-241f61d3fd64 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.418479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a3272-db17-446f-9ef4-241f61d3fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.418794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a3272-db17-446f-9ef4-241f61d3fd64 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.424098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 267/535] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:40.424871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a3272-db17-446f-9ef4-241f61d3fd64 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.446768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6326ff-368b-45f1-bd2c-8ee0d5ccaeda None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.466010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:40.487270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.487984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.488589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.488589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.489375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:40.489752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.491245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b91da3-6fe8-4ab4-be36-67c7527b888c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.504655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 269/536] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:40.506067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.506478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.506931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.530689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:40.531990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:40.563341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a3272-db17-446f-9ef4-241f61d3fd64 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.609680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.610481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.610809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.611466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.620667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.622223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.622882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 270/537] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:40.623949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:40.648042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:40.648966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:40.650623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:40.653656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 268/538] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:40.660111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.740604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:40.818194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c8b1-ecfe-410f-8348-b4aa35b535b3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.844757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e1c0-2572-4084-8c17-7598d140b33a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.858131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.859026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.859453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.859871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.861244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:40.861701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.863516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.869226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:40.878914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 269/539] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:40.886680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.956109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.956749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.957290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.957571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.965900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:40.980632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:40.981725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:40.982692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:40.983234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 271/540] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:40.984335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.998038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6cb7a-11e7-428f-84de-19ef05f48bb6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.999864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:41.038337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.038981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.039261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.039532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.040902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8e2f7189b07f4024b7744322c707c2d5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:41.041207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.042893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.049023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 270/541] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/role_assignments?user.id=8e2f7189b07f4024b7744322c707c2d5&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:41.049810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.157903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d375d-003b-4763-965b-97b33b3313b4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.236899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.237561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.237839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.238109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.254437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d12716-a66a-48f4-9d28-b29c481b2fe7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.260022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:41.264861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:41.265914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:41.266851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 272/542] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:41.272017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.295803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.296732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.297183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.297631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.298491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:41.299084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.300802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.322895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:41.324247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 271/543] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:41.330032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.362943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:41.454541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30784-7428-41f5-b7bd-4937087a9595 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.495729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.496634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.497587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.498056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.499138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=6ebbe8c6f1734885ae15e7137ecf8495) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:41.499756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.501667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.509009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 272/544] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/role_assignments?user.id=6ebbe8c6f1734885ae15e7137ecf8495&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:41.521148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeabdc-99f6-408b-b522-3deabb4738de None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.566401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.567048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.567330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.567606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.568284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:41.568645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.570033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.583201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 273/545] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:41.589147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.731434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:41.743752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648a485-c20f-42c7-8a8d-7b0a42e46558 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.780994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.781846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.782264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.782702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.801149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.802623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c09ae34d-c18a-4d3d-9e76-97449e4d5a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:41.824975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:41.825930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:41.826788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:41.826788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:41.826788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:41.826788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 273/546] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:41.832000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.985413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:41.995197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b6a7-b927-4b39-a523-ead578abdf6b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.050286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f0c6-5c56-47e6-abe4-03c7a0e81af2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.050461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f0c6-5c56-47e6-abe4-03c7a0e81af2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.050780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f0c6-5c56-47e6-abe4-03c7a0e81af2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.052119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 274/547] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:42.052224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.052833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.053115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.053391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.054050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:42.054358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:42.055816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:42.060327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da0bd-649e-4852-8799-a1c9bd618e06 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.060619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da0bd-649e-4852-8799-a1c9bd618e06 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.060958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0da0bd-649e-4852-8799-a1c9bd618e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.068672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 274/548] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:42.081992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.103075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-da0da0bd-649e-4852-8799-a1c9bd618e06 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:42.104288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da0da0bd-649e-4852-8799-a1c9bd618e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:42.207117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 275/549] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:42.267447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce0d4a-d34d-4c3d-b8af-35e68ae0e6f7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.268780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c343f9-bc28-494e-98cc-fa6e0b692c14 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.269236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c343f9-bc28-494e-98cc-fa6e0b692c14 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.269675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c343f9-bc28-494e-98cc-fa6e0b692c14 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.271585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 276/550] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:42.278578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.279103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.279524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.310055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:42.312561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:42.351383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.354436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.354908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.355336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.370297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:42.373971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:42.398948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:42.401455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:42.402763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:42.408783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 275/551] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:42.414517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.437206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:42.448485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 277/552] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:42.491741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.495479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:42.610977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a9d07-e7b9-4080-b370-a9c8e1dc7fcd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.636230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d221b69-eec2-41ce-9330-ac22cfa997db None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.677860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.679272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.680075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.680901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.681336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.681748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.682687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.683110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.684133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:42.685788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:42.688361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:42.704643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:42.709598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:42.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:42.711872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:42.722063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 278/553] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:42.726580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 276/554] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:42.731482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.732567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.796975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:42.817947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:42.845231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720613-4ffa-4177-92a3-93f559bbad72 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.862781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c68372-5577-4565-b7f8-8bf518247f3b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:42.882288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.883134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.883519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.884450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.885250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:42.885654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:42.887300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:42.901697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 279/555] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:42.901964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:42.902601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:42.902838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:42.902965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:42.903151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:42.916533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:42.922852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:42.924057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 277/556] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:42.925273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.012637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5518-ed5a-4352-acf4-507fb74d89f2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.045558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.049619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.050873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.050873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.051188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.068802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ecda-d726-4e1f-80ef-5634e15e9e6d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.103777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.113057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.113770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.114160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.114545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.125863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.127518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.134754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.136793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:43.143534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:43.145148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:43.146217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:43.146217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.146217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:43.152836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 278/557] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:43.153846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.255485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 280/558] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:43.256338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.271577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26bb0f-cbf4-4f28-90c4-a8ad1cee03d5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.286335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.307950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.308811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.309259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.309698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.310364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:43.310699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.312082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69404a3-fbaf-4ab0-9fc1-c05d1cedbf22 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.331452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 279/559] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:43.332370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270935c-894d-4f45-b930-d7dc20c98589 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.332660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270935c-894d-4f45-b930-d7dc20c98589 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.332936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270935c-894d-4f45-b930-d7dc20c98589 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.334424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 280/560] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:43.335161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270935c-894d-4f45-b930-d7dc20c98589 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.372808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc3a6-4895-4a90-82c0-f767a0d563d5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.387153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.410599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.411222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.411496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.411853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.412387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:43.412686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.414030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.427030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 281/561] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:43.432992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270935c-894d-4f45-b930-d7dc20c98589 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.517243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.517884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.518166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.518442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.530550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.532060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-7252f78a-14b3-41eb-94fd-36260702fdd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:43.548206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:43.549396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:43.550546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:43.550546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.550546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:43.553981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 281/562] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:43.555306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.555602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.555954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.556874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.575053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599155a-9548-4e2f-868a-7643a569cf5e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.577911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:43.579162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:43.616226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.617163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.617585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.619272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.633654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.635761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:43.653035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:43.658859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:43.659817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:43.659817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:43.659817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:43.659817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 282/563] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:43.659817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.660936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 282/564] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:43.661363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.688747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.776329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ddcd0-d497-4129-8929-064ac93308d5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.809516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90094-047a-4bbe-bd64-6e93252671df None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.828197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.829169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.829663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.830167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.832527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:43.832846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.834232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.849189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:43.850437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:43.850962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:43.851459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:43.851989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 283/565] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:43.853496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.856068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.864489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:43.866738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:43.879965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:43.881040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 283/566] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:43.883239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:43.949483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:43.966438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560310e7-020f-4b00-b0b8-ad131c022c78 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:43.985458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efb29d-5d2a-4636-95e3-4e6676054e1e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.000716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.001319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.001595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.001866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.002498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.002833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.004179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258e804-d503-4478-9d51-5de1ae8ef67d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.015472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 284/567] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.016615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e4e29-0962-4fa4-8912-80d623810cf9 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.016894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e4e29-0962-4fa4-8912-80d623810cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.017165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e4e29-0962-4fa4-8912-80d623810cf9 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.018337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 285/568] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:44.019056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e4e29-0962-4fa4-8912-80d623810cf9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.024269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.025485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.025978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.026287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.033422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.034875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:44.046823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:44.047834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 284/569] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:44.048729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.107109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.123247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e4e29-0962-4fa4-8912-80d623810cf9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.151671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c66aeb-1c77-4cd5-af29-fec7c07b34d5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.158446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.159071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.159353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.159646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.160365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.160669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.162094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.179886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 286/570] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.180693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.184513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.186300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.187043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.187323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.187593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.198492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-82b497c0-e856-4727-a648-259b51e7baf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:44.203657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:44.204577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:44.205520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 285/571] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:44.206217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.206217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.206346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.224368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:44.225795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:44.283520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458690-38d1-4177-b9ed-8d9ee12f181c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.294102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 286/572] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:44.295040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.326907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.327545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.327827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.328104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.335315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.336761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:44.349001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:44.350223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 287/573] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:44.351192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.363220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.418434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.421784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ed57b-7b79-4f43-92b9-b19b2fc175ee None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.460160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.460684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.460867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fce07-a11e-43a3-9494-482507a87e7d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.460989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.461269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.462047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.463166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.464771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.482411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 287/574] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.483534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.500101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.500895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.501171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.501442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.513035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.514716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:44.528905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:44.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:44.530938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:44.530938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:44.530938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:44.530938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 288/575] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:44.530938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.590942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790e058-af59-48f1-b775-a29d5f7fbabd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.597753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.626044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.626671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.626956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.627233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.627898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.628204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.629566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.638181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 288/576] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.639180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.642564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad225-8b74-4195-bb53-178012632fd1 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.662507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.684369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.685297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.685729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.686006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.686771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.687178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.688497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.706901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 289/577] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.707710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.742548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a450-8c86-4a2b-b1df-56e06242ce3e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.776750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.777589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.777869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.778139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.778889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=87e124d4d97e46b182279f9b24d2052d) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.779196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.780612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.785870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 289/578] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/role_assignments?user.id=87e124d4d97e46b182279f9b24d2052d&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.786627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.818404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7a734-80db-4497-a4c6-26832bbdd87b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.823273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.858556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.859210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.859496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.859776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.860442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:44.860757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:44.862165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:44.876288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 290/579] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:44.877159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:44.909865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad6df-475b-4ca0-ad6f-1d2f7524d1b1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:44.918779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:44.950585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:44.951248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:44.951525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:44.951796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:44.962864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:44.968947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:44.970079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:44.971118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 290/580] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:44.971890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.001109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9bd5cf-b444-4c2f-b712-0bdf0cdbf19e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.037999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.038825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.039180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.039524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.051078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.052514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.067646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:45.067892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.068979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:45.075262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:45.075262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.075262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:45.075262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 291/581] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.075262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.089400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2b00-9754-4d1a-b92c-7380a65da148 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.125310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.125938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.126217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.126492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.139551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.141220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:45.156278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.157341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:45.158439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:45.158439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.158439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:45.164372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 291/582] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.165416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.166181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.187772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd649b-7eb5-466b-8f25-b02440d97df2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.226874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.227521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.227804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.228588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.236219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.237573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:45.249827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.255780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 292/583] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.256702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.282599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661f5f-bebe-475d-9a73-de0cdeb0c0d1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.321659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.322963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.323243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.323509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.324177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:45.324477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.325925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.334084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 292/584] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:45.334904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.345498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.372583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415b91-f63d-4a06-a8b9-cc73031cb197 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.408854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.409748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.410031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.410314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.410991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:45.411365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.412749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.425330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 293/585] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:45.426138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.430926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.454050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa04361-6973-4b88-b2f2-05859fe46058 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.495403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.496125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.496406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.496739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.497399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:45.497706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.499343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.516921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 293/586] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:45.516921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.544850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393640ca-6ac0-474e-95e6-aefb85d00c33 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.581634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.582234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.582519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.582822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.583507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:45.583811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.584982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.585236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.609693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 294/587] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:45.609693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.629011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4116a2-a4f0-4f3f-b412-793e709a1c19 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.663576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.664293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.664640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.664988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.676125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.677705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.686326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:45.693545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.694519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:45.695849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:45.695849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:45.695849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:45.695849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 294/588] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.695849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.734959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38b8a-3ef5-422b-b7a5-ad3f77d56cac None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.770340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.770931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.771210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.771489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.779821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.781703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:45.793926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.795342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:45.796268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:45.799737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 295/589] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.800448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.810318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bbd54-e14f-49da-b7dc-0445fc6164c5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.829187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.854658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.855346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.855629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.856089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.864583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.866093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:45.883855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:45.886122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:45.887187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:45.892746 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 295/590] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:45.892746 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.926977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6a76-77ec-447a-b6b4-5f36594bb549 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.952646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:45.966046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.966747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.967032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.967307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.982823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=8eb093eda2a14771aef59b6269ebe789, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:45.984265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:45.997412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 296/591] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:45 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:45.998202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.009086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c0749-559c-4ced-b796-b7cd41b3e599 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.056667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.057201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.057474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.057799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.058615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:46.058862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.060349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.075945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 296/592] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:46.076842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.125144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.145006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7937c33-6368-400d-9d60-319f1bf3d30b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.197304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.197918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.198203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.198479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.199268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:46.199579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.199788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98160fb8-ee47-4e76-8fcb-4e1ba9ab9cd6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.201016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.235248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 297/593] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:46.236791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.249555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.250872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.251257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.251629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.252424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:46.252826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.255267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.271429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 297/594] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:46.272534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.281467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.378183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6950e-4000-4135-9150-a3b28717ada5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.397534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.403866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad48e-a1af-40e8-a156-ca283e017da8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.431240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.432109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.432659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.433231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.443182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.444394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.445091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.445208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.445556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.445923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.458060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.459778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:46.460884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:46.462043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:46.463755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:46.466669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 298/595] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:46.467581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:46.484287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:46.485328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:46.486214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:46.486214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:46.486214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:46.490488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 298/596] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:46.491484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.591763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.605262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c25e8-8f92-4db2-b422-a8dee92d14a2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.606994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010d356-2538-444c-a5bd-bd5623db9a04 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.651202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.652666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.653087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.653499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.654209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.654562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:46.655024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.655195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.655574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.655940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/f978768101cd4e478b10b2f919383518/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.657071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.674006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=f978768101cd4e478b10b2f919383518, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.676041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.691188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 299/597] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:46 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/f978768101cd4e478b10b2f919383518/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:46.692205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.699569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 299/598] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:46.714395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.727498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.842329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f154622-35e9-400d-b971-fb5568ada446 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.855014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8aa45-7689-4f56-aef0-4d4a85adb0ee None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:46.881043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.881043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.881341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.881525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.893176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.894948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.898221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:46.899057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.899476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.899908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.900859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:46.901309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.902930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899618f-d606-43c9-9f69-c63ca30b282b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fda3027e-197a-4852-b6dd-6f1b71ce9624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:46.910827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:46.911999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:46.912892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:46.912892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:46.912892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:46.912892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 300/599] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:46.913524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0439c-a131-45f0-90d7-9d339a8eff02 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.913810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0439c-a131-45f0-90d7-9d339a8eff02 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.914088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0439c-a131-45f0-90d7-9d339a8eff02 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.916314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 301/600] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:46.920693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 300/601] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:46.921146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0439c-a131-45f0-90d7-9d339a8eff02 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:46.922369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:46.922854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:46.923322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.930989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:46.943458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:46.944702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:47.001784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.023427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 301/602] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:47.024291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.051768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0439c-a131-45f0-90d7-9d339a8eff02 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.090677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.091383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.091798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.092145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.092972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:47.093291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.094986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.113276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 302/603] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:47.114755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.135576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f49097-0ed4-48c6-a947-cfd78fcee0d7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.198983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.207268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.207904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.208178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.208447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.222063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.223542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:47.242044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:47.243589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:47.245673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 302/604] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:47.252905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.322019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967764-ac27-4660-ab24-d0059012c31e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.348630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.365810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.366851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.367356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.367841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.375832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.377537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:47.389414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:47.390330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:47.391254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:47.392122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ba072-413f-412d-b4c1-ac093560c327 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.393973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 303/605] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:47.395196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.428546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.429136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.429423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.429699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.430503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:47.430831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.433003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.445963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 303/606] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:47.446826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.509298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.513157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063289f-17f7-400a-a9be-a78fef8a85c5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.553338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.554336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.554899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.555414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.563739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da2a00-71ab-4b7d-8113-1e93cbf326ff None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.566765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:47.572232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:47.573242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:47.574074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:47.581390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 304/607] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:47.582501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.601422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.601972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.602228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.602507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.603297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:47.604108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.605637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.619397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 304/608] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:47.620199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.629771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.691514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933b9a9-a096-4d3d-acdc-f8bef8b27fd4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.726636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.727365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c5358-22f3-45ec-b2eb-0a7afaa528e5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.727825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.727920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.728288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8e2f7189b07f4024b7744322c707c2d5/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.742249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.744653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=8e2f7189b07f4024b7744322c707c2d5, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.747246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.768523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 305/609] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:47 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8e2f7189b07f4024b7744322c707c2d5/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:47.769907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.773651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28533927-5641-4966-be90-9727cc5134cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.775029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28533927-5641-4966-be90-9727cc5134cf None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.776360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28533927-5641-4966-be90-9727cc5134cf None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.776843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28533927-5641-4966-be90-9727cc5134cf None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.777820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28533927-5641-4966-be90-9727cc5134cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:47.778333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28533927-5641-4966-be90-9727cc5134cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.780007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28533927-5641-4966-be90-9727cc5134cf None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.790099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 305/610] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:47.795889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28533927-5641-4966-be90-9727cc5134cf None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:47.896516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:47.909846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59360b1-47ba-4e2a-9f26-87f0d71939ea None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:47.952484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:47.953441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.954085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.954507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/6ebbe8c6f1734885ae15e7137ecf8495/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.975694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=6ebbe8c6f1734885ae15e7137ecf8495, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:47.977990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:47.998986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 306/611] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:47 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/6ebbe8c6f1734885ae15e7137ecf8495/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:48.006144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28533927-5641-4966-be90-9727cc5134cf None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.048781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.049681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.050108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.050552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.063664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.065415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-fc0f662f-f055-4edb-9544-3d410933f2ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:48.082567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:48.083775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:48.084873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:48.084873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:48.084873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:48.086820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 306/612] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:48.093108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.196582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf16-81e6-4075-a15e-a3bd98778257 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.197681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf16-81e6-4075-a15e-a3bd98778257 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.198368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bf16-81e6-4075-a15e-a3bd98778257 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.200072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 307/613] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:48.205091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.205382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.205654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.231690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:48.233351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:48.237578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a72c5-5896-4c15-b2cc-519b0b0b955b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.258149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:48.274114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.274760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.275037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.275990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.276677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:48.276982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.278415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:48.287122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:48.292673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 307/614] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:48.298099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.320448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 308/615] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:24:48.449544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2f45f-ccc7-4c35-a85a-d35e1008d42f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.497257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.497886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.498165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.498438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.511769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.513234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:48.524949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:48.526752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:48.528163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 308/616] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:48.532514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.645581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.646545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29c93-3363-478f-9d8c-9e7dbb6e29a0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.686672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.686672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.686672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.686672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.687140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:48.687140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.687878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:48.724278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 309/617] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:48.725758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.748255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:48.778988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:48.858500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e0c75-534a-4519-a894-1f56a28ad0ee None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.880590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851265-81a2-4adf-81ae-935d9532b8b6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.935917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.936548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.936828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.937101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.951172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.951808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.952087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.952358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.955580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:48.960954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:48.961952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:48.962863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 309/618] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:48.970664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.980861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.982260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:49.007638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:49.008603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:49.009660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:49.009660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:49.009660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:49.009660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 310/619] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:49.024236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.079417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f2960-0e64-4614-ad14-ae1e6776e856 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.110424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.130909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.131890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.132547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.132843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.133788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:49.134179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.135619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad036603-34fb-4f07-9202-fdf69d7cdd4a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.136124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.147260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 310/620] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:49.151139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe64f2-4b56-41bd-97ab-76d1fde3e4b0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.151451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe64f2-4b56-41bd-97ab-76d1fde3e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.151820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe64f2-4b56-41bd-97ab-76d1fde3e4b0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.154928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 311/621] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:49.155709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe64f2-4b56-41bd-97ab-76d1fde3e4b0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.186129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58592-f1e6-4692-a343-e6996e78f6f8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.230512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.231251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.231481 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.231762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.232578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:49.233197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.234580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6f13-fc84-48bf-bc1d-a6a7fe444988 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.251287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 311/622] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:49.253130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.253579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.253981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.276456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:49.277669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:49.299219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe64f2-4b56-41bd-97ab-76d1fde3e4b0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.359165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.360255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.361118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.361547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.376063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.377685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.378320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 312/623] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:49.379332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:49.394551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:49.396788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:49.398393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:49.398393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:49.398393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:49.398393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 312/624] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:49.404584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.459285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.542453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.569618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd330f5e-7228-4328-9869-d1d7560daa8c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.628501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.629122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.629406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.629682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.630340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:49.630671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.632060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.653900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bdfcf-8a57-4b59-a4c9-785257eb994b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.666328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 313/625] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:49.671951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.705359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.706117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.706401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.706702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.715715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.717257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:49.729367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:49.730313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:49.731248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 313/626] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:49.731741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.780871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.788224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348fc5-85ce-460a-97e0-291abf071f49 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.826376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.826956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.827229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.827499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.836370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:49.837968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:49.848806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22524ed1-631f-4350-b408-caafe7d790b6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:49.850792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:49.853094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 314/627] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:49.855744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.866707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:49.888162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:49.888790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:49.889128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:49.889624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:49.900753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:49.906853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:49.907978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:49.908918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 314/628] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:49.909721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:49.967648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc57633-46d3-46f9-8f1c-5995b7a1429a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.004594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.005542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.005962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.006400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.007327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:50.007772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.009563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.018268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643a3c3-9a7e-42f9-84ab-245f23f5b4cc None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.024656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 315/629] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:50.026264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.044256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.058077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.058875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.059246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.059659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.060636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:50.061030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.062667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.081800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 315/630] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:24:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:50.082700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.119938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.141094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219be25f-be98-4884-a93b-f7cd3382e11f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.185991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.186783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.187586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.187966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.189065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:50.189470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.191657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.197010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48476e8f-629e-4033-b05b-5d28c8384651 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.205665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 316/631] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:50.207216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.238529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.239498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.239947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.240409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/87e124d4d97e46b182279f9b24d2052d/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.256813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=87e124d4d97e46b182279f9b24d2052d, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.258484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.271669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 316/632] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:24:50 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/87e124d4d97e46b182279f9b24d2052d/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:24:50.272286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.307517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.340653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128575e2-feba-48d4-bde5-a9b7ba22887a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.368205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.382794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.383416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.383748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.383971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.388682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473b5e-bb5e-46ab-aa35-1890d4486fba None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.428070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.428701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.428982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.429251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.440909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.442350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-055a81a1-269d-4561-8e7e-50d978422ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:50.458993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:50.459977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:50.460857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:50.460857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:50.460857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:50.460857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 317/633] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:50.461276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb111e-92b8-4ffa-be4d-995ee1e72e2c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.461553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb111e-92b8-4ffa-be4d-995ee1e72e2c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.461828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb111e-92b8-4ffa-be4d-995ee1e72e2c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.463936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 318/634] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:50.465032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59a99b-353f-4569-afd0-0c80e4450894 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.465313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59a99b-353f-4569-afd0-0c80e4450894 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.465668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59a99b-353f-4569-afd0-0c80e4450894 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.466788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.468290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1468001-7ee1-4fc4-a083-f1e0eafdaab9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.472722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 319/635] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:50.473560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59a99b-353f-4569-afd0-0c80e4450894 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.552437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.579537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 317/636] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:50.581011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.581299 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.581574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.590309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59a99b-353f-4569-afd0-0c80e4450894 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.600605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:50.601983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:50.633286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.634208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.634552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.634985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.635857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:50.636288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.637752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8bec2-4fd3-4230-b1ff-120387cb6f65 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.640372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.653653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 320/637] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:50.654867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.655161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.655435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.674633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:50.675838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:50.686508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 318/638] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:50.687670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.756111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 321/639] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:50.763519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:50.854356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225186fb-eced-4423-b0cb-e1e44c3ce469 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:50.868377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:50.896826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:50.897862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:50.898342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:50.898943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:50.911195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:50.913231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:50.933579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:50.934611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:50.935695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:50.935695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:50.935695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:50.939593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 319/640] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:50.946443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.004548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e35fd-d2dd-431f-a8a0-acd0478c9287 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.034824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.045902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.046563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.046881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.047472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.054577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.055988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:51.068204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:51.069197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 322/641] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:51.075000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.102257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f701-3554-47bb-817e-aa30a3daf111 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.140168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.140707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.141044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.141322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.142020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:51.142435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.144024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.160823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 320/642] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:51.160823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.178276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.179286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e63b-2fec-44e6-8e36-48d38cea2563 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.215452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.216225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.216510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.216786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.228120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:51.233937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:51.235179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:51.236063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 323/643] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:51.236774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.274824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1f0e0-4d54-47b4-be96-3e7995d6db9f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.286214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.319168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.319875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.320159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.320435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.333006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:51.339291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:51.340228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 321/644] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:51.341238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.355497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766aeca-ba39-4285-8fb9-8f2e4cd81ec9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.395323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.395858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.396139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.396464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.397146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:51.397452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.398918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.418989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 324/645] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:51.419890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.440489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.469014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942cbf1-fd66-4da7-bc73-e179e35000f1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.508124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.508757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.509037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.509402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.516368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.517823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.529796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6d0e-8427-49d6-9597-802461caaf24 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:51.529990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:51.531031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:51.532139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:51.535967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.536091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 322/646] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:51.537164 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.582116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.582749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.583032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.583308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.583969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:51.584275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.585635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.594403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 325/647] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:51.595568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.652444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93512e-ef5b-482e-bd48-3c412b0d4647 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.687008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.687615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.687896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.688181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.688834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:51.689176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.690611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.695723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.700685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 323/648] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:51.706783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.709356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6f65-9f6c-4566-b02d-2781592c02d2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.744563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.745187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.745467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.745744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.782086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.820937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.823608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.826043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360416b-a95e-4e6f-9473-86419e899d15 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:51.861478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:51.862140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.862350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.862625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.863345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:51.863657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:51.865084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:51.877599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 324/649] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:51.879626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.931786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 326/650] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:51.932895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:51.952972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:51.997519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80509a-bb75-44ef-b869-e51df19a0c1b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.001538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.034006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.034682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.034971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.035250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.042357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655808db-6c0f-423d-8d92-86c277d15a23 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.047550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.049017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:52.063983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.064937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:52.065884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:52.065884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.065884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:52.065884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 325/651] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.065884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.085451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.086395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.086709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.086991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.098609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.100106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:52.117268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.118302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:52.119313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:52.119313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.119313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:52.119313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 327/652] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.119313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.179426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a7109-d120-4f81-bb2d-3416e4cf4830 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.192446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.217726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.218325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.218676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.219212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.219947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:52.220253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.221689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.227804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 326/653] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:52.228575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.229967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4ad22-794b-4012-97b9-ae0672d316da None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.249171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.268616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.269290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.269573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.269843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.282667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.284167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:52.300359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.301415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:52.302317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:52.302317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:52.302317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:52.302317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 328/654] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.303007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.359268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053168-6e1c-4ae7-9de4-c9c88f776622 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.403059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.403674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.403955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.404227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.404874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:52.405272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.406672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3b09e-ffa5-4a8b-892d-0f29e95dafaa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.421150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 327/655] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:52.422314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d43e5-b80f-4c9d-8efc-6148f4bc3fd7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.422605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d43e5-b80f-4c9d-8efc-6148f4bc3fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.422942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d43e5-b80f-4c9d-8efc-6148f4bc3fd7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.424139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 328/656] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:52.424242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c5abc-72cc-43f1-a2dc-70ac341313ff None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.424835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d43e5-b80f-4c9d-8efc-6148f4bc3fd7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.467086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.467865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.467977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.468060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.468134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.468788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:52.469066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.470504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802a092-b926-4129-8f82-85da13aebc32 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.483255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 329/657] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:52.484275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369044d-11be-4469-a844-e213098c8187 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.484557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369044d-11be-4469-a844-e213098c8187 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.484829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369044d-11be-4469-a844-e213098c8187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.506527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4369044d-11be-4469-a844-e213098c8187 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:52.507749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4369044d-11be-4469-a844-e213098c8187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:52.534565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.541427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d43e5-b80f-4c9d-8efc-6148f4bc3fd7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.578309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.578928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.579208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.579481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.580137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:52.580420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.581772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.590758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 330/658] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:52.591739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369044d-11be-4469-a844-e213098c8187 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.605629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 329/659] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:52.606906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.708900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369044d-11be-4469-a844-e213098c8187 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.710682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.720155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0003-1d79-4a3f-9f7b-e62a9ba7c4fc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.752685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.753475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.753755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.754027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.764467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.766334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.768220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.769141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.769419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.769691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.782103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.783834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:52.785291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.786237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:52.787225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 331/660] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.787822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:52.799698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:24:52.801647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:52.810067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 330/661] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.811304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.811696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.908398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83982df6-1e6d-48e6-bf92-eb7b1bb4481d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.926556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb83b1a-ec1b-4f64-92b6-e9eaf4d593cc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.943933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.944563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.944844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.945118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.952679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.954120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:52.965667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:52.966585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 332/662] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:52.967517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.968794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.969474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.969819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.970208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.971003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:52.971451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.972979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.997049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 331/663] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:52.998054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.014117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.060020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.081026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3e767-b070-496b-8020-ce634ee13e80 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.104375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64ed59-0b4c-4b5b-aa18-68da979d5349 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.115240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.115843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.116124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.116495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.117118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:53.117423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.118865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.137719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 333/664] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:53.138803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.146630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.147283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.147563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.147860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.158808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:53.163908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:53.164875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:53.165726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 332/665] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:53.166458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.243786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e583c2-b795-4e3c-80f4-da32ed4847ba None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.247261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.274927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2583dcc-88db-4966-906b-295721b7c252 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.279402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.280011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.280289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.280562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.281264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:53.281569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.283020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.289834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.298679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 334/666] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:53.298679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.325370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.325987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.326266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.326537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.395948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.397442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.414287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50922412-7fc7-44d5-aa08-bcb25508760b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.452364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.453412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.453698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.453976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.471488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.503399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 333/667] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:53.503399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.528803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.533796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.535295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.611913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219d4f-44fe-46df-b28e-1b4a4fcc9729 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.643031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 335/668] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:53.644210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.652096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.654417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.655231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.655699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.656379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:53.656689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.658152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.671527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 334/669] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:53.671527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.716091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.747348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.751652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b236e-cb80-4625-b486-114cebedd86e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.789032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a71fc-2710-487a-81a8-54c7cd54e2cc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.790470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.791114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.791396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.791815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.793027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=f978768101cd4e478b10b2f919383518) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:53.793343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.794810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.810674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 336/670] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/role_assignments?user.id=f978768101cd4e478b10b2f919383518&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:53.810674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.840360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:53.841102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:53.841449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:53.841788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:53.855832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:53.857639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:53.874546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:53.875786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:53.876718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:53.876718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:53.876718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:53.880315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 335/671] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:53.881209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:53.962032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b939a3-5161-4397-89a1-952df4365000 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:53.984884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:53.999644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352fa6f-bc1e-4dfd-a12d-44eedb57b36c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.016515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.017559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.017845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.018404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.033631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.036968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:54.051569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:54.052518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:54.053398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:54.053398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:54.053398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:54.053566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.053566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.053566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.053566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.055369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 337/672] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:54.056640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.065660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.067253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.076410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:54.085642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:54.086807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:54.087788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:54.087788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:54.087788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:54.091192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 336/673] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:54.091192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.181717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd310d21-03c1-4f63-b833-a30f5261cf0b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.223278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eba59d-4164-48dd-ae09-013183a15e38 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.226828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.228533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.228995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.229624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.230897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.231401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.233500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.251739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 338/674] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:54.252888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.267350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.268094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.268606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.268714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.269598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.269983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.271520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.283285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.290796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 337/675] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:54.292598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.355207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.385332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40422-2b4f-450f-a3b1-985a2c1ab70f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.395009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452cd3-64e9-4ddf-8e8f-30d594a59ee8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.424512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.425114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.425407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.425652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.426287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.426587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.427959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122a6-7e82-4343-a77c-06341bcaa01c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.431105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.431953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.432086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.432297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.439400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.440143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 339/676] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:54.440842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.441355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762779c5-8e39-4035-b8e0-7b7e44bb98d3 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.441645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762779c5-8e39-4035-b8e0-7b7e44bb98d3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.441919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762779c5-8e39-4035-b8e0-7b7e44bb98d3 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.443285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 340/677] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:54.443973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762779c5-8e39-4035-b8e0-7b7e44bb98d3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:54.457679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:54.458781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:54.459873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:54.463483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 338/678] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:54.464953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.533473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.559406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762779c5-8e39-4035-b8e0-7b7e44bb98d3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.579520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aa50d-68e3-4c5c-94b9-c5b85a5e6c2d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.593936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.594606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.594877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.595152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.597376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.603403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.605134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-397b36a7-34cd-4677-a9c4-a1af66bb252f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:54.617235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:54.618155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:54.619086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 341/679] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:54.619654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.620650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.620958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.621302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.621928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.622204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.622474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.631133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.632573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.639404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:54.640736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:54.645406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:54.646451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:24:54.647446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:54.651292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 339/680] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:54.652184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.716230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 342/681] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:54.717449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.770784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5fb6-af34-45b9-9747-be36d1222c57 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.793668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.810626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.811264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.811543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.811867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.812604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.812909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.814522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.832188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 340/682] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:54.832188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.858260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12917599-ac61-409c-a196-639a1765e9e3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.890901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:54.906359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.907525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.908881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.909374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.910330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.910919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.912624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:54.930378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 343/683] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:54.931611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:54.941065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca751e-9f57-488c-b227-ed8faa4be5a8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:54.978768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:54.980037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.980319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.980763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.981937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:54.982250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:54.983821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.003827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 341/684] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:55.004537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.041079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2803-0e5c-49b4-a88c-8c941605c339 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.050454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.078498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.079277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.079491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.079703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.080452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8e2f7189b07f4024b7744322c707c2d5) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:55.080766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.082334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.098673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 344/685] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:54 2022] GET /identity/v3/role_assignments?user.id=8e2f7189b07f4024b7744322c707c2d5&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:55.098673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.122974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff45cfa-e306-410d-acb9-af278568193d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.160885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.176049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.176671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.176952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.177224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.177961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=6ebbe8c6f1734885ae15e7137ecf8495) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:55.178272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.180002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.187334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 342/686] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/role_assignments?user.id=6ebbe8c6f1734885ae15e7137ecf8495&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:55.188195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.246462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0cebe-755f-4f1c-8e07-1f2bccc09369 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.296193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.297586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.298057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.298808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.307145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a973d9-acf7-48f1-8df7-a374681c4b5b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.310011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.358512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.359264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.359558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.359874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.375368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:55.380667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:55.381611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:55.382480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 343/687] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:55.389229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.402820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.405379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.469720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.521054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80650bb8-9b47-4357-b90b-bf47dd3a3649 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.539314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 345/688] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:55.541156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.564420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.565293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.565579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.565850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.566700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:55.567334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.569158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.580921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 344/689] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:55.588921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.653930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.693306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b4a3-ee82-4931-b154-52507839b742 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.757064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.757695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.757968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.758237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.777662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.779163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.803550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389f75-201b-406b-870e-881a8ef8b55f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:55.806845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:55.807834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:55.808755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:55.808755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:55.808755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:55.814851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.817808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 346/690] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:55.819861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.880602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.881227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.881502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.881771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.900985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.904876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:55.929659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:55.930696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:55.931589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:55.931589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:55.931589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:55.938280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 345/691] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:55.941222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.006721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16d389-86db-40c0-891a-f4945a1268fa None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.044505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.045118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.045395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.045667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.046317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:56.046614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.048112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6eefa-85a0-49b4-b99b-449dd47c3b8b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.065768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 347/692] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:56.066496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.068188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4db63-12f0-4156-94ad-7ced39a6b056 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.068477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4db63-12f0-4156-94ad-7ced39a6b056 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.068747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4db63-12f0-4156-94ad-7ced39a6b056 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.070557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 348/693] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:56.071411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4db63-12f0-4156-94ad-7ced39a6b056 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.094098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacfde-a1bb-4475-af3e-45cfd1ebb1dc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.132078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.132686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.132964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.133243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.133885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:56.134188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.136198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbec8d-6a66-4d72-b8ba-89c6fcf2b659 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.154448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 346/694] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:56.154448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.154448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.154448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.154930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.170865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:56.172222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:56.208935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4db63-12f0-4156-94ad-7ced39a6b056 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.252558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.253352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.253725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.254102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.254987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 347/695] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:56.256173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.277212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.278623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:56.300188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:56.301307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:56.302202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:56.308282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 349/696] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:24:56.326323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.334248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.414763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad69f-50fb-4473-80ed-ae4ec4418b18 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.488780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.489611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.489991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.490369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.504759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.508060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:24:56.535576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:56.536722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:24:56.537850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:56.542788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 348/697] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:56.549839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e735-4f96-4a15-959f-5598124e13a4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.551690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.582993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.592054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.593015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.593526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.593991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.595002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:56.595505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.597170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.614754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 350/698] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:56.617274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.686123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fce691-f720-4887-b3b1-cc9500d5db76 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.713782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.724663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.725315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.725581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.725851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.726981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:56.727290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.728766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.732526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b206538-363c-42b5-9dec-c1d392743a0f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:56.752857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 349/699] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:56.752857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.782864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:56.783471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:56.783749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:56.784020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:56.784809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=87e124d4d97e46b182279f9b24d2052d) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:56.785607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:56.787425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:56.802684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 351/700] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/role_assignments?user.id=87e124d4d97e46b182279f9b24d2052d&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:56.802684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:56.905555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:56.962055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1584c-19a0-4dc9-8c2a-96c862b44da0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.018519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5920c1d-ada3-467c-a2fa-7e117c840d95 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.040257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.040903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.041182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.041550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.064735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.066220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:57.087783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:57.090474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:57.091537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:57.091537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:57.091537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:57.094171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 350/701] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:57.104375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.105879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.106439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.107894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.108003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:57.151558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:57.232729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ea51-ac5c-4980-98f0-7504bda8bc8f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.274089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.275572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad381c-5d2d-4711-9563-255c3c442301 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.276122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.277867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.278478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.279036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.279881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:57.280414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.282307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.301400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 351/702] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:24:57.304292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:57.380186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:57.455560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 352/703] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:24:57.457085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89b7b-787f-4f60-86f1-17089f4042af None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.457646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89b7b-787f-4f60-86f1-17089f4042af None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.457778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89b7b-787f-4f60-86f1-17089f4042af None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.459301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 353/704] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:57.465271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d89b7b-787f-4f60-86f1-17089f4042af None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:57.493909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:57.503061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4779c-2364-40a6-86ed-56c7620c2dd1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.551138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.552383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.552682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.552943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.553600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.556348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878110e-ddf5-47e6-b57d-fa6f9bd6087f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.581707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 352/705] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:24:57 2022] POST /identity/v3/users => generated 254 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:24:57.583266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e340c35-e94e-4c95-bf4d-b8d5300c395c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.583556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e340c35-e94e-4c95-bf4d-b8d5300c395c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.583827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e340c35-e94e-4c95-bf4d-b8d5300c395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.608811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d89b7b-787f-4f60-86f1-17089f4042af None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.610178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5e340c35-e94e-4c95-bf4d-b8d5300c395c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:57.611700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5e340c35-e94e-4c95-bf4d-b8d5300c395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:57.659515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.660164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.660443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.660716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.676307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.678516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.682442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-553587c4-ea07-491d-81c7-4c1d8f0653cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:24:57.705140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:57.706230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 354/706] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f23df-d0a7-4a01-b504-befc14b1a803 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f23df-d0a7-4a01-b504-befc14b1a803 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f23df-d0a7-4a01-b504-befc14b1a803 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.707169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 355/707] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:57.711011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 353/708] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:57.715713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f23df-d0a7-4a01-b504-befc14b1a803 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:57.716654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.717171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.717647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.755257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:57.757767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:57.851375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 354/709] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:57.862370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f23df-d0a7-4a01-b504-befc14b1a803 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:57.903686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:57.904292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:57.904569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:57.904882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:57.905529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:57.905889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:57.907328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:57.926297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 356/710] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:57.927652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:57.943778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.013528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.048479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970db20-1e50-4a53-964d-8af23d323131 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.076420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.104027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.105713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.106229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.106696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.124049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.125736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:24:58.151216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:58.153134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:24:58.154035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:58.163771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 355/711] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:58.164687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.165954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff94f64-f410-4646-92f5-20f84515bdf1 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.208976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.209603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.209878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.210148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.210759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.212132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:58.224341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:58.225415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:58.226450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 357/712] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:58.227601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.268627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.308389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e024cf-f35f-4399-9989-f21a47e730ef None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.352683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.353349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.353629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.353904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] PATH_INFO: `/v3/services/network` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.354605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.356977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.359031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a36b9-138e-4884-a387-44d4dcda36be None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:58.368519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:58.369587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:24:58.375706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 356/713] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:24:58.376544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.397024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.416222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.417282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.417776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.418293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.419219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.419763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.421567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.440447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 358/714] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:58.441806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.484098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383983c3-0cc2-4e07-a6a1-3f2b6dc85500 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.521120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.521636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.521919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.522193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.522859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.523166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.524552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.526723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.535403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 357/715] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:58.536412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.562339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a6e02-9415-425b-8bd8-3d96dff62db9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.604252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.606068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.606768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.607239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.608156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.608746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.610360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.618735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 359/716] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:58.619864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.640863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e17d19-27f4-4403-b189-060e86ee0390 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.672137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.675920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.676690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.677133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.677434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.678342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.678940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.680677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.690783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 358/717] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:58.690783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:58.762431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:58.773201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11389c4-687d-4933-a45d-73539e8770e8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.810133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c635daa-ddcc-48a0-8dd6-68cdb9df324a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:58.826752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.827440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.827745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.828058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.830682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.830682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.830682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dab0d-d98b-4dbf-a812-31c54147f137 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.844953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 360/718] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:58.870180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:58.871908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:58.872256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:58.872548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:58.873266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:24:58.873573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:58.875021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca9957-9e6f-4ede-acfa-0e46a4c754f5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:58.882152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 359/719] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:24:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:59.365218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826e5d-23b6-4ca8-980b-1230fb382df0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:59.365733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826e5d-23b6-4ca8-980b-1230fb382df0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:59.365841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85826e5d-23b6-4ca8-980b-1230fb382df0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:59.367421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 361/720] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:59.372940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:59.373350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:59.373544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:59.392783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:59.393969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:59.488944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 360/721] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:24:59.811752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826e5d-23b6-4ca8-980b-1230fb382df0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:59.934468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85826e5d-23b6-4ca8-980b-1230fb382df0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:59.943593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:59.970241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:59.970899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:59.971181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:59.971461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:59.991523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-68293b95-166e-4003-903e-0ce78bd08718 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:24:59.997058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:24:59.998006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:24:59.999263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 362/722] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:00.004290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.090989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:00.137971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d1473-5329-49e3-8874-772e5c613483 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:00.178104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:00.179105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.179619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.180101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.180963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:00.181461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:00.183041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:00.192431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 361/723] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:00.197849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.198427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa09e64-4fde-4ebf-91c3-64858bd74747 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.199287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa09e64-4fde-4ebf-91c3-64858bd74747 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.199573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa09e64-4fde-4ebf-91c3-64858bd74747 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.202693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 363/724] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:00.213480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.213773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.214054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.255735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:00.256922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:00.317443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc582ee8-6f39-4fd3-9a2f-0fbcefe72545 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:00.363262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:00.390293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:00.391754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.392253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.392740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.405075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:00.406939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:00.420868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 364/725] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:00.422924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:00.423845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:00.424729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:00.424729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:00.424729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:00.426687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 362/726] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:00.433745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.567314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:00.571286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284f5e8-73c0-46cb-a6eb-8b2a6fccb16f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:00.616043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:00.616997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.617485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.618014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.618924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:00.619460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:00.621552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:00.633849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 365/727] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:00.640109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.752955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ada-87dd-4626-bf0f-a7562f26e8d8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:00.756192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.829159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:00.830174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.832112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.832603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:00.847519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:00.850573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:00.873051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:00.874038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:00.875010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 363/728] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:00.879693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:00.915931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebaae-ac1b-4350-95d9-64bc3e07de69 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:00.943061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:00.979572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:00.983781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:00.984321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:00.984801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.005436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:01.014244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:01.015282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:01.016115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:01.023465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 366/729] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:01.030038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.062369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.085993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea047e5-3aa0-4d34-a9a9-a31108efb416 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.146502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.147055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.147330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.147599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.148329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:01.148630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.150049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b8e-4fe7-4894-b565-306b6d187714 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.172164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e703-4b34-4c5d-a141-a2d8a710c5dc None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.175355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 364/730] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:25:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:01.176668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a78-eb7c-4bcc-aa79-8c096a332ea3 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.176956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a78-eb7c-4bcc-aa79-8c096a332ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.177234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a78-eb7c-4bcc-aa79-8c096a332ea3 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.182604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 365/731] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:01.184556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.184842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.185115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.204231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:01.205528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:01.215531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.216767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.217194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.217610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.219032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:01.219541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.221608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.234203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 367/732] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:01.239374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.298841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 366/733] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:25:01.299895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.337634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.354692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.374126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a557f-32e3-4b00-92ce-c78f6e675ff9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.418791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.419331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.419635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.420162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.453394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522f43e-1e8f-46c6-a991-c80f300de9c5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.489418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.490026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.490306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.490799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.501555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.503023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.504468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.505915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f26e7-0052-4f1d-828a-eff6bf363294 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d790809f-486f-4d90-bc3a-22ca2ff7a7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:01.518364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:01.519594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:01.520472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:01.520472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:01.520472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:01.524121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 367/734] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:01.525638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7829e7-1f77-4248-80c0-36989530ef4b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.525924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7829e7-1f77-4248-80c0-36989530ef4b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.526195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7829e7-1f77-4248-80c0-36989530ef4b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.527741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 368/735] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:01.528676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2160-35e3-459a-ad02-6528c7af26fc None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.529021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2160-35e3-459a-ad02-6528c7af26fc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.529299 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2160-35e3-459a-ad02-6528c7af26fc None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.534508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 369/736] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:01.534508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2160-35e3-459a-ad02-6528c7af26fc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.598535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.604828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.625157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 368/737] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:25:01.626635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.626911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.628144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.638605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2160-35e3-459a-ad02-6528c7af26fc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.648296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:01.649492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:01.674584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.675305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.675579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.675843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.676488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:01.676789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.678154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a10ad-b234-4edc-8fbd-c929128907f4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.691780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 370/738] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:01.693204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.693498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.693772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.711840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:01.713025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:01.722156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 369/739] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:25:01.723029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.792347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 371/740] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:25:01.792347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.835883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f13b0-4e29-48f3-812f-1117855768de None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.848004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.860097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:01.876251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.877084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.877375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.877658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.889572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:01.894668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:01.895603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 370/741] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:01.901703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:01.913361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708b39b-5ca0-4128-ab62-882bf3101cbf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:01.951339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:01.952301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:01.952858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:01.953430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:01.964285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:01.966767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:01.988569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:01.989577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:01.990482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:01.990482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:01.990482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:01.997298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 372/742] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:01.998880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.014048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca5e40-8322-41f3-92ba-f670d2fce387 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.051624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.052268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.052528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.052796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.060673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.062102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:02.072905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:02.073939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 371/743] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:02.075745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.107829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.110213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a21d-6b41-496b-8054-2049b28bec5a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.124021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.145374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.145969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.146251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.146524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.147213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:02.147515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.148886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.157439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 373/744] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:02.158296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.179769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec44e-5975-44a0-ab3a-0ce239aa7c73 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.213573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.214104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.214382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.214735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.216016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:02.216322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.217701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.229947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 372/745] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:02.230857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.259829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813c21-0c2e-4539-9081-de333dbe842a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.301859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.302466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.302770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.303042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.303882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.305282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.328701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 374/746] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:02 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:02.329330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.333536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.348072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93110d8-4504-4400-a2b2-3f01328d7ebb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.379872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.400780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.401453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.401735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.402063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.403365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.405466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.430677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 373/747] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:02 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:02.430677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.469881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e828bf-343a-4caa-b1dc-7cdf6262e3cb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.537135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.537854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.538142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.538418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.539251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:02.539565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.541048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.566871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 375/748] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:02.568238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.575990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb44ff-3a53-40fc-9719-b87277ddeedb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.629702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.630343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.630617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.631071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.642567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.644034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.655273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:02.662583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:02.663592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:02.664489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:02.664489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:02.664489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:02.664489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 374/749] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:02.664489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.687913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.698474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954bc45-6fe6-4aa7-8ba7-ae6d987ba02f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.735784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.736327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.736603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.736872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.745455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.747511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:02.758859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:02.759961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:02.761248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 376/750] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:02.761940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.800841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc987f52-93cd-46c7-937b-94bb83f5acb2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.845341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.845953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.846233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.846505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.895492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157485cd-792a-4020-8797-43d919e7acdf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:02.917709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:02.925983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.927528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.950111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:02.951148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:02.951682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:02.952718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:02.953867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:02.954380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:02.956864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:02.976781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 377/751] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:02.978320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:02.993126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.043417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 375/752] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:03.044733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.096706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a687d1b-b5ac-4f01-ac7c-94c0a7a11282 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.132506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.132921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.133202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.133477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.134202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:03.134473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.135941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.149618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 378/753] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:03.150053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.150514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.156186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d212d-fc11-483d-b2ea-5fd08f877493 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.192738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.193700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.194164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.194618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.202873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.204564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:03.217219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:03.218176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:03.219181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:03.224775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.226174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 376/754] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:03.227385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.259678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca21f-6fc4-4b38-b3f3-fcb3ac299bd6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.296080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.296943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.297386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.297811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.309078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.310798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:03.328141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:03.329167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:03.330060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:03.330060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:03.330060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:03.336514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 379/755] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:03.337562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.341348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638446-2390-4710-825c-b443c0d3f0af None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.381126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.381742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.382026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.382306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.383078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=87e124d4d97e46b182279f9b24d2052d) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:03.383392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.384824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.391971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 377/756] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/role_assignments?user.id=87e124d4d97e46b182279f9b24d2052d&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:03.392911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.421731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.460951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c512-b368-42e2-8f06-9b110c238f3d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.488617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.514477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b7ca5-0050-4bca-bfc4-f7f7342718ff None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.520061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.521095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.521590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.522072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.523243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:03.523788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.525628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.552497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.553122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.553407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.553687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.554354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:03.554686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.556098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.561891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 380/757] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:03.576226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 378/758] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:03.577046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.582810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.682973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.705869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7782a-764b-4931-ab08-77fdd876a417 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.745399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.746429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.747082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.747215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819374c0-8fa2-4999-8260-0d44a4e584e6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.747778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.755506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.757390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:03.770835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:03.771751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:03.772650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:03.775232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 379/759] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:03.784023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.789493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.789638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.789852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.790100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.813684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.815193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:03.835052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:03.836069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:03.838095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:03.838095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:03.838095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:03.838095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 381/760] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:03.847681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:03.883093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:03.931552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd06a3-9f4b-4876-b77b-abad82648376 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:03.974447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:03.979483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:03.980100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:03.981274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:03.982192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:03.986865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:03.988670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:03.993362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118392a2-b369-43ca-b23f-c3ee5a216b9a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:04.011636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 380/761] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:04.017075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:04.020256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:04.035731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:04.036465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.036595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.036804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.037456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:04.037762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:04.046694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:04.052145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 382/762] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:04.057489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:04.153182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:04.162755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a274f8-f7e3-4290-b0e2-85578a4ac212 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:04.200854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:04.201490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.201776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.202052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.209836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:04.211365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:04.223307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:04.224240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:04.225543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 383/763] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:04.231257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:04.273313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4455cf-fbb7-462b-a64e-5a29b69bce03 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:04.334367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:04.335086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.335369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.335649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.336386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=fbfd83920ceb42598c2f2ba9b008a94a) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:04.336697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:04.338125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906136b7-ea50-4cd0-ba44-4d7a15c5d692 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:04.354787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 381/764] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:04 2022] GET /identity/v3/role_assignments?user.id=fbfd83920ceb42598c2f2ba9b008a94a&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:04.377376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd32ab-2375-4fc3-84fe-0ec916e670c2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:04.419427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:04.420046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.420324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.420594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.421297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:04.421604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:04.423087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:04.429070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eeef-d728-4e85-81c2-368508d43b3d None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.429358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eeef-d728-4e85-81c2-368508d43b3d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.429629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eeef-d728-4e85-81c2-368508d43b3d None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.431374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 382/765] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:04.437009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 384/766] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:04.437342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062d462-0337-4f51-bcc6-b2a79540f639 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.437629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062d462-0337-4f51-bcc6-b2a79540f639 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.437932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062d462-0337-4f51-bcc6-b2a79540f639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.442627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:04.475063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:04.487530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0062d462-0337-4f51-bcc6-b2a79540f639 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:04.488738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0062d462-0337-4f51-bcc6-b2a79540f639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:04.563664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91009f36-8d0e-475e-a3a6-0e0a8f793af0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:04.603574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:04.604439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.604727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.605030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.608611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=8eb093eda2a14771aef59b6269ebe789, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:04.610071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61368d-f622-4c55-b675-fab69c81c7a5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:04.631504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 385/767] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:04 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:25:04.640341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:04.657131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 383/768] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:04.938562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac33cd2-b55b-40d5-a0f1-0ccd7f12fdf4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.938944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac33cd2-b55b-40d5-a0f1-0ccd7f12fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.939128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac33cd2-b55b-40d5-a0f1-0ccd7f12fdf4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.940680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 386/769] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:04.945950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:04.946245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:04.946530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:04.956822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac33cd2-b55b-40d5-a0f1-0ccd7f12fdf4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:04.970167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:04.971778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:05.095006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.106191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 384/770] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:05.145000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac33cd2-b55b-40d5-a0f1-0ccd7f12fdf4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.196500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.197049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.197329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.197603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.198191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.199619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:05.211375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:05.212385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:05.213816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 387/771] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:04 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:05.218521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.330676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79346e85-f71b-40b5-afc5-680c019ba3cd None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.359099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.369145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.369763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.370039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.370310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.370992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] RBAC: Adding query filter params (name=network) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:05.371296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.373228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.378376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 385/772] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:05.383872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.494037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.512338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ce3e-7fca-4989-b623-19fba9facdc5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.551704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.552359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.552686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.552966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.553600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] RBAC: Adding query filter params (type=network) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:05.553903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.555299 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.562102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 388/773] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:05.562607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.568340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.701081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb02ab-053c-4b3d-aabe-ffd677bf0a2a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.719016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00f20-f734-4077-82b4-bd2de8e0b30f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.760833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.761470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.761750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.762052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.762659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.763726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.764224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.764334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.764614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.764892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.765610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] RBAC: Adding query filter params (interface=public, service_id=a4ae77da0e3146b7b5fa29f3f42a6c57, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:05.765924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.767319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:05.773443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:05.774437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:05.775945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 386/774] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:05.782665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 389/775] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/endpoints?service_id=a4ae77da0e3146b7b5fa29f3f42a6c57&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:05.787819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.791018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:05.870039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.893696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.941250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526915f-12d5-452b-892b-90bf3d8f452d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.942491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae72c0-2e04-41e5-b8e2-30e2f04e876b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:05.979648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.980252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.980529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.980799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.981924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.983681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be748b83-2962-4ccd-96c1-745801fdb06d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:05.988582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:05.994701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 390/776] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:05.994701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a93e3-13ad-49b9-ad14-3a5abfa84ed7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.994701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a93e3-13ad-49b9-ad14-3a5abfa84ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.994701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a93e3-13ad-49b9-ad14-3a5abfa84ed7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.994701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 391/777] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:05.995227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:05.995227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:05.995640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:05.996114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:05.996766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:05.997928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:06.000346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.000624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.000894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.011237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 387/778] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:25:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:06.017648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:06.041556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:06.043008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:06.174925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c763e-f799-498b-8a45-368ffd9c9210 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:06.184292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 392/779] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:06.194173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:06.214790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:06.216456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.216975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.217449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.218348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:06.218882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:06.220471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:06.226291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 388/780] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:25:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:06.232524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:06.294814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:06.345716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae67eb-28e2-42d6-8bc9-efcb5880fce8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:06.382285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:06.382937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.383219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.383495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.384201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] RBAC: Adding query filter params (interface=public, service_id=88a75d8ae0da42ee9c2995ff13bf0200, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:06.384514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:06.385887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:06.392183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 393/781] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:06 2022] GET /identity/v3/endpoints?service_id=88a75d8ae0da42ee9c2995ff13bf0200&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:06.397651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:06.524727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef60fe-209a-4a3b-a81d-e5bdcd0fd3a2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:06.570668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:06.571316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.571602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.571882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.572546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:06.573916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee033e-58f8-4def-9803-18103640ac10 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:06.583513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 389/782] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:06 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:06.695568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:06.857656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b835-db43-4458-957f-7fbcd5d3d368 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.858621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b835-db43-4458-957f-7fbcd5d3d368 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.859155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26b835-db43-4458-957f-7fbcd5d3d368 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.860700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 390/783] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:06.866994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.867536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.868017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.922793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd4dc6-a876-4a18-b2e6-5458502913cb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:06.931675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:06.933201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:06.951681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:06.971707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:06.972576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:06.972985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:06.973384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:06.998466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:07.004121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:07.005105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:07.005985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:07.009274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 394/784] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:07.015783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.046902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:07.119971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 391/785] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:07.170538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9cee-840a-4b98-9e8b-72316ce8bc25 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:07.207135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:07.207745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.208032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.208297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.209032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:07.209341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:07.210756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553de63-abcd-4be5-90b9-98710c4011e1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:07.221854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 395/786] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:07.227665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.262333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42cf7-5808-4b5b-8a1b-201589ba9ca8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.262333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42cf7-5808-4b5b-8a1b-201589ba9ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.262333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42cf7-5808-4b5b-8a1b-201589ba9ca8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.262333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 396/787] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:07.267623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.267908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.268184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.296236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:07.297434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:07.351865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:07.405389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 397/788] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:07.437491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dffb51-3b40-4524-a42f-ac9e224e75b7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:07.482388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:07.483069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.483352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.483631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.502687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:07.502687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:07.517075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:07.526068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:07.527367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:07.529129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:07.529129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:07.529129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:07.531321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 392/789] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:07.537475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.598399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:07.693513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd6f36-d9c2-49e1-b459-01b69671efda None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:07.718775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d46bb7-20d5-4bc3-a874-379ef4eb889b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:07.745445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:07.747122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.747832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.748727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.751228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:07.751964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.752334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.752707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.753887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:07.754199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:07.755886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:07.763622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:07.769028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:07.770258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:07.771420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:07.773843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 398/790] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:07.774849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.827194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:07.890816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2429b-e49f-4e32-abee-23c5f17f0465 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:07.898568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 393/791] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:07.899300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.928865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:07.929821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:07.930265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:07.930715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:07.942827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:07.948769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:07.949823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:07.950750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:07.953503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 399/792] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:07.954550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:07.991938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.005317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61902a-15c7-478e-9bc8-91eb2826737a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.040264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.040987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.041326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.041666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.042563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.042802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.044592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.057611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 394/793] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.058635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.067098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.071382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6b87b-da26-475d-a86d-5af93ddea75d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.110901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.111913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.112346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.112818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.121341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.124045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:08.136454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:08.138152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:08.139504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:08.141469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 400/794] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:08.142506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.166812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e0730-04ce-4d69-9850-f4559c33f401 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.202468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.203164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.203538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.203910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.204668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.205141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.206515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.215067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 395/795] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.216132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.232789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.257908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ad006-b614-4d6c-b93e-96a8d9adc069 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.295019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.295901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.296355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.296820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.308847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.311271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.320527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389ed6-0a38-4ae1-bdba-fd219aa81a56 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:08.328410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:08.329579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:08.330719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:08.330719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.330719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:08.332533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 401/796] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:08.333553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.343900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.366347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.367136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.367671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.368047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.369197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.369621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.371544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.394341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 396/797] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.395562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.450816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a129d5-78da-4d5b-ba08-64b507e6a75d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.466303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.491451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.492453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.492966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.493392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.499686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b363-995d-4549-93bb-9c1041515baa None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.506574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.508408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:08.525183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:08.526376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:08.527571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:08.527571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.527571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:08.530335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 402/798] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:08.531331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.538188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.539019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.539400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.539776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.540527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.540933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.542378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.560490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 397/799] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.561909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.597941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.643091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ea965-f80a-4b06-8967-58995e9c6b7e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.673879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70ddc-e07c-4481-b85e-785b0c687ade None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.679037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.679669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.679950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.680224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.691491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.692936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.704538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:08.708138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:08.709240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:08.710546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:08.710546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:08.710546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:08.713968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.714144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 403/800] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:08.714805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.714949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.715334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.716515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.717278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.717679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.719172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e59875-835c-4827-b451-d889b2ab3e11 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.736674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 398/801] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.736674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928dc5-1ab2-46b3-a940-8d970f4d2f1a None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.736674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928dc5-1ab2-46b3-a940-8d970f4d2f1a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.736674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928dc5-1ab2-46b3-a940-8d970f4d2f1a None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.736674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 399/802] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:08.737100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7928dc5-1ab2-46b3-a940-8d970f4d2f1a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.840439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5d12b-6251-4d4e-bd2d-732827b24d78 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.848197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7928dc5-1ab2-46b3-a940-8d970f4d2f1a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:08.853113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.880891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.881538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.881809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.882085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.884272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:08.885295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.885295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.885486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.886035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:08.886339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.888330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.891773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:08.893298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:08.905904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 400/803] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:08.905904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-79ad2717-579b-4c0d-bcbe-0a499f728b89 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:08.906505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:08.907443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 404/804] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:08.908543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:08.909382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:08.930008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:08.931238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:08.966819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:08.999713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 405/805] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:09.000555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.016025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ef193-3bfc-4319-a94e-a9c8fabf0624 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.051519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.052236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.052487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.052762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.059964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.061399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:09.072886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:09.074052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:09.076151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:09.079472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 401/806] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:09.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.114769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.119027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3f9c-2682-44fa-8835-2a20d58eed3d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.156099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.156815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.157162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.157501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.164854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.166488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-61f114c1-4f0f-401a-b8b7-ee31b6091526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:09.179420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:09.180471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:09.181827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 406/807] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:09.182327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de46737-32d2-40a0-b484-042f42d05a42 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.182519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de46737-32d2-40a0-b484-042f42d05a42 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.182807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de46737-32d2-40a0-b484-042f42d05a42 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.188436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 407/808] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:09.188436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de46737-32d2-40a0-b484-042f42d05a42 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.191884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce6609-9be0-437c-82ef-4d3a953049fb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.203051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.227870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.228406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.228687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.229226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.229954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:09.230261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.232521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.253933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 402/809] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:09.254767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.301203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de46737-32d2-40a0-b484-042f42d05a42 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.342359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.342997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.343275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.343550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.344280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:09.344640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.346086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1159a-06cb-426d-b2d6-fba35b756370 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.357799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb4468-47eb-4452-a52c-ce0de1ce29e9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.364025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 408/810] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:09.365314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.365593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.365865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.382735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:09.383902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:09.394334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.394981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.395254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.395523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.396211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:09.396511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.397423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.397906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.411721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 403/811] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:09.413493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.442095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.457357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 409/812] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:09.458194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.521192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27a731-5d1c-46d8-9d4a-39c78c7bd422 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.572255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.573225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.573683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.574122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.579782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c311158-7316-4fd7-86ce-a0bd5070a34a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.589864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.592547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:09.616061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:09.617245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:09.618298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:09.618298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:09.618298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:09.618298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 404/813] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:09.619569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.637192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.638842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.639717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.640251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.641265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.643311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:09.655541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:09.656524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:25:09.657377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:09.658131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 410/814] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:09.658206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.675785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.712934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.738300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abaf4c2-5a37-43e3-9a9e-13aff3f99ae9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.770815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ee7fe-e602-4a3e-8041-cb392fe148e6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.786123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.786693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.786980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.787257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.805355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.806005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.806284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.806557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/87e124d4d97e46b182279f9b24d2052d/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.808466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=87e124d4d97e46b182279f9b24d2052d, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.809984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.822256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 411/815] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:09 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/87e124d4d97e46b182279f9b24d2052d/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:09.823341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:09.868916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.870417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:09.906705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.944599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893c71-c409-4ac2-bcb6-e3d8a35acaf2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:09.974843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:09.986128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:09.987115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:09.987565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:09.987947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:09.989039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:09.989465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:09.991734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.003563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 405/816] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:10.010823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.012199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 412/817] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.014274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.116326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b054a7a-21a4-4644-bf4a-638816c09698 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.126495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a121-933b-4f26-9ccd-28e39f1899c2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.154602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.155483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.155937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.156213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.156848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] RBAC: Adding query filter params (name=network) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.157153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.157879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.158512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.163416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 406/818] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.164207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.168588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.169255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.169535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.169835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.170829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.173069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:10.182186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:10.183306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:10.184154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 413/819] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:10.186092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.227350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.285761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84fc-0779-4403-a420-37469d4fd211 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.290687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73befc3e-3a51-44ea-b673-91d28f5c4b95 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.325614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.326821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.327102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.327370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.330249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.330916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.331220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.331478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.334516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.335988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.342807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.344299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:10.347980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:10.348859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:10.349771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:10.355084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 414/820] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:10.355990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:10.361555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:10.362515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:10.363485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:10.363485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:10.363485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:10.367488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 407/821] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:10.368133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.425518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.471671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.475092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35475c-de7b-492b-91dd-f268335e0ae4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.477726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228ff7-5a00-42e5-89e1-4e06e7c983ca None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.511503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.512110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.512390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.512663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.513835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] RBAC: Adding query filter params (type=network) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.514136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.515527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.516349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.517075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.517397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.517678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.518310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.518617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.519997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.521493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 415/822] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.522531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.534765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 408/823] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.535650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.629355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3ee0-6a6f-49e9-ba7a-8c494860ca6c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.641546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9e536-4e91-4b64-9a41-e9b11c803418 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.656985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.674428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.675065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.675340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.675612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.676307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.676614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.678038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.681720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.682329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.682610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.682918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.683560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.683926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.685320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.695555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 416/824] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.695555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.697660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 409/825] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.698364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.721236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.813915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5850c7-e5b8-4b82-94bc-7651718fa090 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.816303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038603-851c-42e7-8308-c97a02e863ef None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:10.850814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.851415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.851696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.851968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.852626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.853509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:10.853993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.854578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:10.854985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:10.855262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:10.855900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:10.856191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:10.857952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:10.863229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 410/826] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:10.863820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.885458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 417/827] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:10.886600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:10.956835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.963267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:10.985987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2f7ca-ad29-4948-8d95-43242c9ae10d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:11.035637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:11.036381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.036665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.036942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/fbfd83920ceb42598c2f2ba9b008a94a/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.058162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=fbfd83920ceb42598c2f2ba9b008a94a, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:11.059667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:11.080606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 411/828] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:10 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/fbfd83920ceb42598c2f2ba9b008a94a/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:11.080797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:11.180042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d03d-4e1f-4a83-90a6-a37e719124c6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:11.204147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd26ac-7352-49a9-b3ab-507c6cd53bcd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:11.220890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:11.221976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.222485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.223028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.232542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:11.234814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:11.239995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:11.247337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:11.248853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.249409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.249735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.250431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:11.252192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:11.252893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:11.253799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:11.254719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:11.262251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 418/829] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:11.264661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:11.285776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 412/830] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:11.389285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:11.395780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01a60b-3b40-4085-9e28-ae0ec12168ad None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:11.443428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:11.443959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.444236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.444506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.445201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:11.445504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:11.446955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec257f85-f756-4e26-8e42-0f867c529238 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:11.460624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 419/831] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:11.466434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:11.577174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd27b2-b477-4fb2-bdd6-ba20f03e1784 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:11.613321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:11.613927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.614254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.614536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.615301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:11.615609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:11.617037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f54a-3d45-46a8-8f99-4ad1a87ee9d2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:11.623996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 413/832] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:11 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:11.786278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91f1d4-3179-4e65-802e-0b8e07f890b8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.786676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91f1d4-3179-4e65-802e-0b8e07f890b8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.786880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91f1d4-3179-4e65-802e-0b8e07f890b8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.788458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 420/833] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:11.794103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:11.795065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:11.795392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:11.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:11.816550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:11.867980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:11.894797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 414/834] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:12.162727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91f1d4-3179-4e65-802e-0b8e07f890b8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:12.274174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91f1d4-3179-4e65-802e-0b8e07f890b8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:12.310892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:12.311534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:12.311810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:12.312078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:12.323601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:12.328561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:12.330767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:12.332218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 421/835] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:12.339732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:12.450999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3569406-ab4f-4a4b-9ccd-d64e3e9a1cc4 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:12.486949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:12.487554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:12.487830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:12.488099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:12.488758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:12.489058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:12.490410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:12.502749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 415/836] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:12.507721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:12.523527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:12.613275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6ba40-f17b-487f-95dd-c01fa6920b97 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:12.652851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:12.653878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:12.654464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:12.655136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:12.665856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:12.667636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:12.683379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:12.684462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:12.685986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:12.685986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:12.685986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:12.688093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 422/837] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:12.694700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:12.741370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:12.804947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27394ce2-980d-423c-be6a-ee79f11c6c26 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:12.841746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:12.842604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:12.842982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:12.843266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:12.843993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:12.844301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:12.845862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:12.866277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 416/838] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:12.867622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:12.934460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:12.979199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449bc08-8298-4454-b324-36766aabaad5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.015710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.016336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.016615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.016887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.024199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.025632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-50b3d68b-c7e9-442a-a0af-8e63ddae835c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:13.037838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:13.039800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:13.042120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 423/839] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:13.045397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.133427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a3e01-82f1-43f2-ab09-2e1ea621de48 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.133770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a3e01-82f1-43f2-ab09-2e1ea621de48 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.133951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a3e01-82f1-43f2-ab09-2e1ea621de48 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.135507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 424/840] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:13.142584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b2b6-7bc3-4cc6-bed5-0d9a78014d12 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.142890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b2b6-7bc3-4cc6-bed5-0d9a78014d12 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.143633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b2b6-7bc3-4cc6-bed5-0d9a78014d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.155168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfc473-ad93-4c0d-9728-470dae73a3f5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.156806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.166468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b2f7b2b6-7bc3-4cc6-bed5-0d9a78014d12 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:13.167674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b2f7b2b6-7bc3-4cc6-bed5-0d9a78014d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:13.200011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.200826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.201186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.201470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.202212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:13.202517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.204170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.227377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 417/841] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:13.235026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.243413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 425/842] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:13.302423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c47-6349-45ef-8b18-895a843b7980 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.302756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c47-6349-45ef-8b18-895a843b7980 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.302929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c47-6349-45ef-8b18-895a843b7980 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.304490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 426/843] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:13.309279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.309562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.309840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.340145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:13.343044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:13.351528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbc74-9b2f-4a75-a246-2b43c7b7c12e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.391179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.391883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.392238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.392546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.416650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.428368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.447083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 427/844] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:13.476846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.478341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.522831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.598373 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 418/845] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:13.604918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.644153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f760f5b-6e08-49bb-92a4-979982826cd4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.670151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.693926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.694860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.695569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.696492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.711313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:13.715569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc85138-d5ef-4d63-8a77-f3987147c067 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:13.717813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:13.718971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:25:13.720003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:13.723240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 428/846] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:13.730886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.755403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.756025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.756301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.756566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.766449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.767896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.776408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:13.783384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:13.784438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:13.785369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:13.785369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:13.785369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:13.785369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 419/847] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:13.785369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.847272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c60c90-68b1-44c3-88ce-10ababec6e36 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.883666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.884294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.884580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.884848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.885508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:13.885810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.887209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff2b94-6073-47ed-8622-9e67a1a7b3eb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.895248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 429/848] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:13.896399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae847d-2c95-43f5-b572-a61af90cae4d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.896684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae847d-2c95-43f5-b572-a61af90cae4d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.896962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae847d-2c95-43f5-b572-a61af90cae4d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.898421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 430/849] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:13.899155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae847d-2c95-43f5-b572-a61af90cae4d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:13.900284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f5cb-5597-4f01-b6c9-307be718de7a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:13.914285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:13.936691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:13.937330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:13.937626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:13.937903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:13.938490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:13.939880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:13.948405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:13.949619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:13.950780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 420/850] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:13.951839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.007333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae847d-2c95-43f5-b572-a61af90cae4d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.014358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.042309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.042935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.043212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.043484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.044210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.044538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.046495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd55b7-881a-4a3d-bd2f-652fcfc3c2f8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.052490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b2784-fe8a-468b-89e3-0e9364a033ba None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.057490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 431/851] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.058667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.058980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.059260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.077209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:14.078397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:14.086457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.087107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.087386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.087660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.098685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.100120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:14.114298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:14.115378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:14.116263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:14.116263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:14.116263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:14.116263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 421/852] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:14.116506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.144649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 432/853] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:14.145488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.149959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.228414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfa274-cbce-47c3-b83f-2967d473df09 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.243229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.250606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f91f7-75ad-400b-86e4-a90ae76dcee6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.267590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.268427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.268881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.269372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.270192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.270757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.272288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.276834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 422/854] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.277897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.289796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.290665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.290963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.291234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.298949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.300388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:14.311882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:14.312950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:14.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:14.317879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 433/855] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:14.318764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.388128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.399057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637421e5-89f8-4766-ba45-b3fc9a32e734 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.428853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67134a-5c3b-44d8-aac8-24d7ba90a10c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.433870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.434475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.434792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.435071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.436335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.436640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.437999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.449332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 423/856] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.450118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.463681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.464359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.464679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.464955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.465590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.465898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.467883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.473990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 434/857] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.474816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.512043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.570016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2975e-ecb7-4eeb-8dff-83861ec0fc86 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.606808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e900e2-a5bc-42e8-a73d-1a898e13a7d7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.615930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.616841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.616841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.617130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.617762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.618069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.619523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.632594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 424/858] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.633432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.646578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.647233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.647511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.647782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.648364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.649768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.652151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:14.659641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:14.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:14.661540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 435/859] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:14.662336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.755339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746226e6-a4c0-4183-9539-0cde2c2930eb None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.760980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.780182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927591fb-9d3c-46ee-89f7-3c9ed27b8f8d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.791292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.791925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.792204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.792473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.801653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.803130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:14.815305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:14.816372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 425/860] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:14.817316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.822705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.823324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.823601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.823868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.824597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=87e124d4d97e46b182279f9b24d2052d) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.824901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.826720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:14.843016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 436/861] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/role_assignments?user.id=87e124d4d97e46b182279f9b24d2052d&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:14.844499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:14.915183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:14.953397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1d148-7087-4577-91a9-0156748ed714 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.983209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d26b1d-2db6-4ad6-a7db-89e72a6fd238 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:14.996066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:14.996690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:14.997072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:14.997257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:14.997927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:14.998470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:14.999990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.004765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 426/862] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:15.010587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:15.027555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.028169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.028450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.028728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.030043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:15.030350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.031878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.049948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 437/863] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:15.057568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:15.066352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:15.134467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff87f6-910f-4d82-93b1-a1c2f98801a4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:15.177900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.178863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.179424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.179709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.180353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:15.180664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.181423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595bd0b-b42e-45ff-bc14-141130ea0f2c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:15.182150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.185496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:15.189666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 427/864] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:15.224403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.225035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.225312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.225584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.306824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.308395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.318757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:15.418349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 438/865] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 15:25:15.424086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:15.527198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb124c6-212d-4e33-a959-bc36d38fac0c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:15.563340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.563972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.564251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.564521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.575439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.577384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:15.592429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:15.593387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:15.594273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:15.594273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:15.594273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:15.594273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 428/866] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:15.599493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:15.709890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3874-2e7f-4d0b-9df1-3169b3fa89c0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:15.718743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:15.747614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.749246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.749755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.750227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.751077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:15.751585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.753124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.764852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 439/867] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:15.774676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:15.825912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:15.888338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97d3e8-e8eb-48c7-a277-4b323950106b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:15.925591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:15.926225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:15.926506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:15.926814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:15.934139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:15.936249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:15.948683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:15.949583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:15.950470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 429/868] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:15.956034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:16.061856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b1a17-22c6-4fb6-8dde-c012fc7b2c63 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:16.096633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:16.097556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.098048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.098513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.099441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:16.099938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:16.101555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd532-fab1-4ed9-b8df-6ae15c076293 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:16.115189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 440/869] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:16.122678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:16.231754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afbd9b-9d53-463d-90e5-233cc8d00dc2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:16.249175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:16.272744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:16.273350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.273631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.273908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.274635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=fbfd83920ceb42598c2f2ba9b008a94a) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:16.274969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:16.276389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e567c-cc3f-4bfe-935b-b58cdcd2328e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:16.283508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 430/870] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:16 2022] GET /identity/v3/role_assignments?user.id=fbfd83920ceb42598c2f2ba9b008a94a&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:16.495614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4c392-c225-40d6-b6b9-81c7d2c7304d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.496115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4c392-c225-40d6-b6b9-81c7d2c7304d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.496115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4c392-c225-40d6-b6b9-81c7d2c7304d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.497767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 441/871] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:16.503545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f236-4652-4b19-b204-65912addb5c6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.504001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f236-4652-4b19-b204-65912addb5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.504082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256f236-4652-4b19-b204-65912addb5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.523412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d256f236-4652-4b19-b204-65912addb5c6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:16.524611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d256f236-4652-4b19-b204-65912addb5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:16.602095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 431/872] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:16.870850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4c392-c225-40d6-b6b9-81c7d2c7304d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:16.899727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:16.902301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b91219-be5d-41b6-9a6e-28fb2f5b3583 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.902582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b91219-be5d-41b6-9a6e-28fb2f5b3583 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.902890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b91219-be5d-41b6-9a6e-28fb2f5b3583 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.904283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 432/873] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:16.910218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:16.910799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:16.911289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:16.931896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:16.933828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:16.992757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4c392-c225-40d6-b6b9-81c7d2c7304d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:17.010734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 433/874] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:17.028609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:17.029221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.030022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.030297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.031110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:17.032494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba8472-5fa9-4af8-83fe-1e27579d0fad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:17.054831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 442/875] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:16 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:17.281237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:17.281761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d0d90-5d89-4ea9-a72e-ebc2268ed437 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.281987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d0d90-5d89-4ea9-a72e-ebc2268ed437 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.282266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d0d90-5d89-4ea9-a72e-ebc2268ed437 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.283904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 443/876] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:17.288995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.289281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.289563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.310558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:17.312148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:17.326066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:17.410103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6836e4-e6bb-4448-b17a-f3c8b3348b1f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:17.419973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 444/877] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:17.452715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:17.453645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.454171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.454623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.455412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:17.456947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:17.466138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:17.467315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:17.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:17.470761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 434/878] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:17.479966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:17.496342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:17.593455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08223-33e4-408a-885d-3e644d1c0f6f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:17.629530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:17.630134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.630416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.630719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.631360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:17.631667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:17.633035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:17.638040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 445/879] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:17.643787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:17.699688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:17.749814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77732b9c-2886-4981-b015-4fce52364143 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:17.758979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:17.781941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:17.787803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:17.788447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.788726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.788996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.789632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:17.789936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:17.791523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aa70c-db6f-4f3e-abcc-fd8ba63094cf None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:17.796715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 435/880] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:17.806819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afffc-86f0-445a-9593-8962dd04a686 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:17.848047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:17.848585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:17.848868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:17.849145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:17.849820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:17.851269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8024b-79cb-4ae3-b1c8-ce7166fca1ca None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:17.872080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 446/881] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:17 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:18.412633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e807252-85cb-422e-80c0-37e313654743 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:18.412993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e807252-85cb-422e-80c0-37e313654743 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:18.413252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e807252-85cb-422e-80c0-37e313654743 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:18.415376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 436/882] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:18.420408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:18.420845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:18.421241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:18.440856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:18.442050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:18.523381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 447/883] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:18.795018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e807252-85cb-422e-80c0-37e313654743 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:18.904604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e807252-85cb-422e-80c0-37e313654743 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:18.943202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:18.943838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:18.944117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:18.944457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:18.955268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-1ba0b5bc-53c3-4afb-b570-5f67d6660745 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:18.960419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:18.962496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:18.964540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 437/884] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:18.972010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.043891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.080006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319971-cfe7-4fda-8e27-e088b6868dfe None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.080287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319971-cfe7-4fda-8e27-e088b6868dfe None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.080477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319971-cfe7-4fda-8e27-e088b6868dfe None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.082897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 438/885] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:19.088184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.088468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.088741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.097547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c873412-5714-4402-91dc-c0ab211d5c0f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.109021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:19.110215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:19.129952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.141059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.142185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.142590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.142901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.143818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:19.144130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.146198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.156480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 448/886] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:19.162518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.191053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 439/887] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:19.292614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94b4df-7171-469a-9a38-253c8cc9fc1f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.328511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.329528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.334890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.335407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.335850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.348073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.349707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:19.365393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:19.366573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:19.367757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:19.367757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:19.367757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:19.369451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 449/888] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:19.375126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.487228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a1c73-5f84-4f15-bc7c-748dd3b00b25 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.511186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.525131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.525738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.526014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.526285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.526990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:19.527277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.528762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.547680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 440/889] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:19.548724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.575046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.646596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389950da-5b34-4209-8090-74c5b185e420 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.668939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602153dc-116a-4be4-9237-0d8c1bf42d89 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.682342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.683050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.683330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.683604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.684254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.685802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:19.695332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:19.696340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:19.697187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:19.701546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 450/890] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:19.702945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.710404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.711061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.711339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.711613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.722163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.724075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:19.738702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:19.740532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:19.742186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 441/891] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:19.745800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.755484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.824407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5168ac-a727-4993-a506-b938c3ad14c5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.839109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:19.863278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.863882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.864171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.864435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.865071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:19.865372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.866755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43743994-fe00-45a5-96cf-823fe93e53d3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.867752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd0b5-50a8-407f-a4bd-a77755e53c08 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.872574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 451/892] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:19.876615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a372ace-8711-43a6-b0d2-96d9e0052cb2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.877604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a372ace-8711-43a6-b0d2-96d9e0052cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.877885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a372ace-8711-43a6-b0d2-96d9e0052cb2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.879750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 452/893] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:19.880364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a372ace-8711-43a6-b0d2-96d9e0052cb2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:19.915794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:19.916315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.916592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.916862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.917588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:19.917888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:19.919344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45972b-37f6-4fb7-b68c-51a05ff2568a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:19.937265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 442/894] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:19.938711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c0c1-4376-425c-baeb-b50c038fbcca None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:19.939002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c0c1-4376-425c-baeb-b50c038fbcca None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:19.939277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844c0c1-4376-425c-baeb-b50c038fbcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:19.957190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2844c0c1-4376-425c-baeb-b50c038fbcca None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:19.958391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2844c0c1-4376-425c-baeb-b50c038fbcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:19.992373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a372ace-8711-43a6-b0d2-96d9e0052cb2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:19.994744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.027517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 443/895] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:20.027844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.028407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.028763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.028860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc71c-fe4d-452d-9d9d-bec1b3f57219 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.029036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.029129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc71c-fe4d-452d-9d9d-bec1b3f57219 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.029313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc71c-fe4d-452d-9d9d-bec1b3f57219 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.029669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:20.030476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.031512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 444/896] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:20.031938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5ed1-2818-45c2-9117-a660c32a4656 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.032093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fc71c-fe4d-452d-9d9d-bec1b3f57219 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.040940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 453/897] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:20.042017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.042305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.042567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.061085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:20.062270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:20.107967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.146673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 454/898] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:20.146673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.170614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fc71c-fe4d-452d-9d9d-bec1b3f57219 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.213136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.213907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.214290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.214717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.260120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.267396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649188cf-b884-487a-813f-af21e41b0bf2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.289818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.291589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.314100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.314827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.315106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.315368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.318398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] RBAC: Adding query filter params (interface=public, service_id=f296ad7083c74f26922e2e8620fe78ec, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:20.318398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.318398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.323085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 455/899] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/endpoints?service_id=f296ad7083c74f26922e2e8620fe78ec&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:20.323844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.368905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.413181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 445/900] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:20.414671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.426624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e23f-2ee3-4a55-a41c-30c2168d6700 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.461653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.462247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.462524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.462900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.463540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.464898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.479003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 456/901] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:20 2022] POST /identity/v3/services => generated 241 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:20.480096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.510035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.531449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc424-a0e1-4a61-8dac-6913e329d3cd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.577899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.578915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.579384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.579663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.580282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.581676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.588055 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 446/902] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:25:20.588919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.618116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b02b-ea6e-485b-81ac-686c07f2278e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.627050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.669160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.670586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.671449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.672249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.686331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.687801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:20.708079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:20.709176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:20.710078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:20.710078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:20.710078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:20.710078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 457/903] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:20.716273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.739686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef123b5-2cb5-4c65-9249-3d8f48b3d3ba None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.785961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.786605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.786913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.787189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.787783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.789159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:20.802672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:20.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:20.804555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 447/904] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:20.811819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.830341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.865231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad43efe-317d-482b-8d0e-b0ba63914b7c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.907532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.908127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.908411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.908685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.909336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:20.909639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.911201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.926521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8203f02-48d0-4de1-b10c-f75223aab4e2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:20.929237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 458/905] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:20.930666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:20.969511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:20.973082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:20.973980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:20.974455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:20.975204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:20.977155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:20.977673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:20.979790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:20.988096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 448/906] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:20.994831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:21.057160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84038-0564-476a-9c55-5dffe1687a73 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.109197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.109870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.110151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.110426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.113637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:21.115128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e02-6976-401c-bd5a-205acc97fd2e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.121962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.124266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:21.140822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:21.141757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:21.142684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:21.146811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 459/907] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:21.148824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:21.160679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.161291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.161568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.161842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.162477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:21.162806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.164165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.171108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 449/908] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:21.176290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:21.203803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:21.269444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4eaa5-5e17-48f0-aae0-12f64baff5d6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.306395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.307390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.307809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.308344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.309510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:21.310037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.312515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.316781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30074b98-453f-4483-81f6-5a78ba82e5df None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.327927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 460/909] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:21.336335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:21.365816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.366854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.367382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.367867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.368896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] RBAC: Adding query filter params (interface=public, service_id=f497547f38a34cfbaf41c2c7b66c9c04, region_id=RegionOne) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:21.369511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.371125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.380116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 450/910] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:21 2022] GET /identity/v3/endpoints?service_id=f497547f38a34cfbaf41c2c7b66c9c04&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:21.386301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:21.407249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:21.455778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4aa87-0cf9-4c5b-8ca4-f975940d7246 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.470735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:21.492251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.493016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.493394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.493763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.494583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] RBAC: Adding query filter params (role.id=ab45b3a2512744b8948b506ea9dac763, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=fbfd83920ceb42598c2f2ba9b008a94a) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:21.495038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.495659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c82483-02ba-492a-a26b-d2f889c83c47 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:21.496678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693333d-b8cf-4a79-b82f-040200ba9455 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.503625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 461/911] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:21 2022] GET /identity/v3/role_assignments?user.id=fbfd83920ceb42598c2f2ba9b008a94a&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=ab45b3a2512744b8948b506ea9dac763 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:21.541480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:21.542468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:21.542995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:21.543469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:21.544295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:21.546214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f417c9-55cc-45a7-8ee0-7db9426c8898 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:21.553963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 451/912] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:21 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:25:22.515044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c96f3-3166-4584-8760-fe9c702697d2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:22.515660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c96f3-3166-4584-8760-fe9c702697d2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:22.515660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c96f3-3166-4584-8760-fe9c702697d2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:22.517181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 462/913] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:22.522783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:22.523218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:22.523401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:22.541976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:22.543214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:22.599656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c8cf-9251-4753-b301-84e1f51ed5e1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:22.599938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c8cf-9251-4753-b301-84e1f51ed5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:22.600129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c8cf-9251-4753-b301-84e1f51ed5e1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:22.601698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 463/914] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:22.607153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:22.607786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:22.607786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:22.620529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 452/915] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:22.626361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:22.627584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:22.716255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 464/916] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:22.901081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:22.972761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:22.997036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.009601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70c36f-6b1a-4ea1-b1ba-722137d15f5a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.053938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.060517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.061163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.061950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.062257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.063348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.065445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:23.078985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:23.080069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:23.080922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 453/917] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:23.086074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.115068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955b4b2-300e-4bb5-a8aa-2fbcba5d602b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.154075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.155089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.155911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.156560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.157377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.158988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:23.168371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:23.169382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:25:23.170229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:23.178480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 465/918] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:23.179687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.197046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396607a-8a63-4a8e-94f6-4b3b99bce961 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.235109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.235711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.235988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.236261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.236946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:23.237241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.238697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.240474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.249724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 454/919] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:23.255633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.293451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69e33-a95e-4c13-9e65-b493323a15fa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.308092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.332829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.333730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.334215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.334718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.335577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:23.336085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.337670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.348242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 466/920] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:23.349397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.365287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafc435-e988-47af-a33f-97f57ffb3a03 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.402178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.402857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.403158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.403408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.404040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:23.404401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.406344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.416671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 455/921] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:23.417730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.459518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b6b53-8f4c-425c-a11d-ba4027d6b269 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.487526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.497029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.497979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.498464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.498973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.500352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:23.500940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.502557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.512928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 467/922] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:23.514047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.532784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ec548-901b-4c05-aab7-5800bae78079 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.558417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.566949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.567540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.567814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.568082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.568782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] RBAC: Adding query filter params (interface=public, service_id=3cf23395920840e5a513ae58d8fab1c0, region_id=RegionOne) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:23.569115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.570465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3053e3-0a0d-4f3a-844b-782d74a38d9c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.575291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 456/923] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/endpoints?service_id=3cf23395920840e5a513ae58d8fab1c0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cec3a-326a-4597-81ec-de0186772dc2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cec3a-326a-4597-81ec-de0186772dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cec3a-326a-4597-81ec-de0186772dc2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 457/924] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767cf7-1b4a-477a-b259-219347dfe2af None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.580360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767cf7-1b4a-477a-b259-219347dfe2af None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.580799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767cf7-1b4a-477a-b259-219347dfe2af None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.581174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 458/925] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:23.581851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767cf7-1b4a-477a-b259-219347dfe2af None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:23.623311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf567d5-37f2-4fea-8522-112661ea2026 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.657138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.657662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.657936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.658205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.658884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.660252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e8616-0571-44c9-9a63-f7c2429c9fbf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.680518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 468/926] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:23.682043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.682333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.682607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.702517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:23.704101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:23.707148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767cf7-1b4a-477a-b259-219347dfe2af None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:23.738046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:23.751115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:23.751800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.752196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.752306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.753910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:23.755375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce4716-3531-4d56-a2f9-ff47ba698b9b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:23.770684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 459/927] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:23 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:25:23.770684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:23.770684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:23.770684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:23.797117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:23.798365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 469/928] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:23.798482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:23.880155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 460/929] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:24.073646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.101719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.169141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.177519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb264a6-399a-4770-8a0d-e62b91cd6d43 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.214489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.215163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.215438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.215710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.216295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.217652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:24.227846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:24.228827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:25:24.229724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:24.232734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 470/930] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:24.239533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.289879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7cea2-d55f-4e63-b8ff-2fc8d5aebdfd None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.308943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.338247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.338843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.339119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.339389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.348302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.351068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:24.356070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:24.357281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:24.358735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:24.362830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 461/931] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:24.364020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c887c-57d1-4b9c-aa85-3bf299206aef None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.364150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.403387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.404364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.404847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.405343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.407153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:24.407698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.409493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.420160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 471/932] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:24.422092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.473218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c69b8-d220-4e0e-8ebc-09bb71d74e68 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.510796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.511397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.511673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.511943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.512607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:24.512907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.514272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.529059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 462/933] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:24.530283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.538352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e908dec-d572-4ac0-8a6b-c407590542fa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.575473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.583231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.584611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.585230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.585735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.586586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:24.587127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.588921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.598912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 472/934] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:24.601503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.623161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.638290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28beef50-902c-4906-8ce1-604126aa640a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.674148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.674845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.675142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.675641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.686192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.688082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:24.706573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:24.707587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:24.708502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:24.708502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:24.708502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:24.712604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 463/935] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:24.713819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.724756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8cefc-5f86-45f2-a8b9-2e45d55bd63f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.764346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.766101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.766600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.767223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.768096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.769869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.805946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 473/936] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:25:24.817771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:24.830194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732356-13e5-4bdf-a135-996fc34bb9f7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:24.878746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:24.879274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:24.879555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:24.879826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:24.880488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:24.880792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:24.882284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:24.896733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 464/937] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:24.903807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:24.990374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:25.017953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d332-a097-427e-864d-9bd1f45cad56 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.053660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.054285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.054562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.054860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.062722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.064163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:25.079565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:25.080567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:25.081512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:25.085468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 474/938] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:25.086826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:25.196664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ddcdf-cb1a-4270-a8fc-65941af11eb6 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.232622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.233230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.233510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.233841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.235220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:25.235640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.237078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.255855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 465/939] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:25.261243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:25.374907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33ea6-0ebe-40f6-b99c-feec2fd521d1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.410821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.411455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.411736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.412010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.414003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:25.427448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.429068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:25.444148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:25.445312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:25.446408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:25.446408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:25.446408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:25.447908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 475/940] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:25.454419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:25.463680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:25.566759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94acd4fb-a188-4702-ba50-1fbbfc815028 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.603647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.604299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.604516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.604790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.605446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:25.605749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.607519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56acd6c-978d-4a1a-b614-49d937971ea7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.620293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 466/941] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:25.626186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:25.694625 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2143-2c0c-415b-991d-3ecf96acdb9c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.694970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2143-2c0c-415b-991d-3ecf96acdb9c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.695251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2143-2c0c-415b-991d-3ecf96acdb9c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.696639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 467/942] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:25.701888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b687-78dd-4928-a0c7-c1ba66c8afaa None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.702171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b687-78dd-4928-a0c7-c1ba66c8afaa None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.702458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b687-78dd-4928-a0c7-c1ba66c8afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.722336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e7a9b687-78dd-4928-a0c7-c1ba66c8afaa None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:25.723669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e7a9b687-78dd-4928-a0c7-c1ba66c8afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:25.740026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c068-c113-4ab5-aefa-beaa3dc6ae22 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.775039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.776020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.776510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.776988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.784696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.786396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.795086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:25.798288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:25.799221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:25.800268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:25.802534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 476/944] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:25.806750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 468/944] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:25.814685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:25.822550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551c544-e4b4-4a8f-92fb-758c80ad4627 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.822884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551c544-e4b4-4a8f-92fb-758c80ad4627 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.823278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551c544-e4b4-4a8f-92fb-758c80ad4627 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.829192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 469/945] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:25.830395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.830711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.830986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.851846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:25.853070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:25.867924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:25.929872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866528b3-2767-4cc7-b99c-ca5b5e78e65c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:25.939142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 470/946] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:25.967355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:25.967881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:25.968686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:25.969196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:25.969995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:25.970305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:25.971847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:25.998809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 477/947] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:26.002854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.111443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba36414-5d32-40a2-89f2-9ebdc87e6826 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.121171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.139897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.149968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.150628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.150932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.151209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/fbfd83920ceb42598c2f2ba9b008a94a/roles/ab45b3a2512744b8948b506ea9dac763` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.153421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=fbfd83920ceb42598c2f2ba9b008a94a, role_id=ab45b3a2512744b8948b506ea9dac763)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.154924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.182669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 471/948] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:26 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/fbfd83920ceb42598c2f2ba9b008a94a/roles/ab45b3a2512744b8948b506ea9dac763 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:26.238187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.246887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cf1e0-25e2-4416-8d0e-f9c68c8698c2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.255797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.288557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.289183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.289461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.289740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.290406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.292923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.294922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING keystone.server.flask.common [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 15:25:26.322488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.sql.core [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:25:26.322488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 15:25:26.322488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [parameters: {'id': '5824fea0ad9f4470b4229c741e6a2304', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 15:25:26.322488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5824fea0ad9f4470b4229c741e6a2304.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5824fea0ad9f4470b4229c741e6a2304. Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:25:26.342127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 15:25:26.343162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application yield resource Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise self.value Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application session.commit() Aug 24 15:25:26.345383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:25:26.346695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:25:26.347886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:25:26.348829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [parameters: {'id': '5824fea0ad9f4470b4229c741e6a2304', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.350165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:26.351178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5824fea0ad9f4470b4229c741e6a2304. Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 478/949] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 15:25:26 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:26.353432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.398541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701a957-c9e0-4c10-8879-352cb8c15350 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.438503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.439462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.439929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.440463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.441307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.442935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:26.453193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:26.454595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:26.455721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:26.458356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 472/950] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:26.467730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.485841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506666d-dcfa-49a5-bf19-79d3619931a4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.528015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.535509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.536248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.536957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.537486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.548916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.551280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:26.567958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:26.568942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:26.569969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:26.574110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 479/951] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:26.575902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.599105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9022601-5752-405d-a5f3-998cebf65b5e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.612937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.638120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.639578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.640101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.640635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.641888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:26.642495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.644233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.654687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 473/952] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:26.660735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.722811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58380-3a29-417b-a437-7890f6033801 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.771574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.772206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.772741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.773027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.773702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:26.774011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.775778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.798699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60762-e71c-4490-a500-6e445acecd61 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:26.806789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 480/953] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:26.819875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.843976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:26.844828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.845109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.845382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.846093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:26.846401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:26.847905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bd9ae-43a4-48cf-b213-f5fea3670934 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:26.854005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 474/954] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:26.860693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:26.914382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e3f5-3e6d-4009-9b18-13571063dc57 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.914763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e3f5-3e6d-4009-9b18-13571063dc57 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.914970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e3f5-3e6d-4009-9b18-13571063dc57 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.916464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 475/955] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:26.922282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:26.922914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:26.923418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:26.942805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:26.944369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:26.959427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:26.981451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b89bec-edd1-46ce-9e4f-fb8843647a9a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:27.026367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:27.027010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:27.027286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:27.027553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:27.028199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:27.029560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:27.033252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 476/956] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:27.059015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 481/957] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:27.328387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:27.436157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86ce88-f5aa-4cda-98ee-f7ea3787243a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:27.447090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:27.480845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:27.481478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:27.481759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:27.482029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:27.482892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:27.484475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:27.493525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:27.494524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:27.495418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 477/958] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:27.501607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:27.609333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7913-f237-44eb-a84a-2aa4a1343413 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:27.645848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:27.646848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:27.647337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:27.647817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:27.648676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:27.649183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:27.650762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0750b3-2e4d-441f-a027-a7aac3e44547 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:27.656402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 482/959] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:27.664413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:27.772399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f733c-18a4-43ca-96bd-71c1c13cb261 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:27.808678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:27.809278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:27.809552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:27.809820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:27.810450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:27.810772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:27.812126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db6bfc-dba6-447b-89c5-20f9b5537a1d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:27.817499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 478/960] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:28.210480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701102-dca7-48cc-9246-62972de8b760 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.211063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701102-dca7-48cc-9246-62972de8b760 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.211063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71701102-dca7-48cc-9246-62972de8b760 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.212656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 483/961] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:28.217919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.218482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.219257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.238135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:28.239713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:28.269970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:28.318764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 479/962] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:28.588030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71701102-dca7-48cc-9246-62972de8b760 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:28.591402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:28.700786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71701102-dca7-48cc-9246-62972de8b760 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:28.737152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:28.738179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.738721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.739218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.750745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:28.755757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:28.756720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:28.757582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:28.760197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 484/963] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:28.768061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:28.875679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e15a57-91a4-4a42-8e57-d8e89c30ca5d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:28.911539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:28.912165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.912442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.912708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.913397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:28.913696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:28.915141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0ef4-a383-49f7-88cc-fbd06cd5380e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:28.924913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 480/964] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:28.930341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:28.946278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb47c-a880-4ab1-97d5-3b4f591c3def None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.947097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb47c-a880-4ab1-97d5-3b4f591c3def None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.947589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb47c-a880-4ab1-97d5-3b4f591c3def None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.949991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 481/965] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:28.955405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:28.955920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:28.956369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:28.977600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:28.979114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:29.003037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.003830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.055074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaaaaa-d533-4147-b1f1-f465e8522cfa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.065874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 482/966] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:29.095279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.095971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.096251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.096529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.108492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.110034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f913da30-f1c9-47ec-b2c7-ecfbece4bbc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:29.124374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:29.125909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:29.127315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:29.127315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:29.127315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.130785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 485/967] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:29.132535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.134265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8c29-e096-4fef-876e-e2efa548b47e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.134549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8c29-e096-4fef-876e-e2efa548b47e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.134860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8c29-e096-4fef-876e-e2efa548b47e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.136439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 486/968] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:29.142392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.145536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.145536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.162205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:29.163726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:29.242143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 487/969] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:29.262775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6077c32-1f17-4f78-889a-a6c7dceae997 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.300353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.301239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.301663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.302081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.302913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:29.303435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.304987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.317382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 483/970] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:29.323138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.363346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.378944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.407007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.441594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03b2a6-4a04-4970-9493-24fb7f2c390a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.467855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e57821-f68e-4d3b-a60d-f7086d013dc4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.479429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.480367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.480853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.481336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.489775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.491679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.503760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.504370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.504648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.504921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.505571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:29.506287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:29.507404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:25:29.508387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.508971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.509542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING keystone.server.flask.common [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 15:25:29.515463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 488/971] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:29.516665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.522448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.sql.core [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:25:29.522448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 15:25:29.522448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [parameters: {'user_id': '36dbd7d2e9024894a37ad4c95c2cde53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 15:25:29.522448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81446) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result.read() Aug 24 15:25:29.530834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:25:29.532643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application yield resource Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise self.value Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application session.commit() Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:25:29.533629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:25:29.534528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:25:29.535482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application result.read() Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:25:29.538190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [parameters: {'user_id': '36dbd7d2e9024894a37ad4c95c2cde53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:29.539311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 15:25:29.540365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 484/972] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 15:25:29 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:29.541531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.623452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.635337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc7b2d-8e94-450e-a409-ff8c859d8b1a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.639079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.650874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c3948-d7d3-44b4-a7c2-ed31ff144929 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.674998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.675737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.676015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.676402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.677037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.678414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.684219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.684826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.685106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.685380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.686118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:29.686425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:29.688150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:29.689967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:29.692292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.693902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.696757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 489/973] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:29.696757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.716125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 485/974] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:29.717247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.813199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc326f1-e946-401c-bb91-1eb6c18142b1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.823177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2dae34-35d4-4fa6-a27d-db13c014abfa None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.847163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.847794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.848075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.848345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.855682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.857027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:29.857148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.857892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.858178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.858465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.859108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:29.859447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:29.860949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:29.862063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.866507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 486/975] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:29.867318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fa9fb544-5be1-4a98-bb15-8f2a82127768 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:29.868612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:29.869547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:29.870484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 490/976] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:29.871125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb115-8efa-42f4-94c6-e47a357a25d9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:29.871486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb115-8efa-42f4-94c6-e47a357a25d9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:29.871773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb115-8efa-42f4-94c6-e47a357a25d9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:29.873012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 491/977] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:29.873701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cb115-8efa-42f4-94c6-e47a357a25d9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:29.903536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:29.973396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552457-b983-44d2-9cd7-78f134a46628 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:29.975737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cb115-8efa-42f4-94c6-e47a357a25d9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.009214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.009821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.010097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.010226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.010366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.010816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.011024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:30.011117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.011320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.011414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.012674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.017878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 492/978] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:30.018702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.094168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.097565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.099386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bead58-154f-4e8e-8caf-c688790794b7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.121505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.131016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6f00-53ac-46b0-9583-5b6646b9629f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.164161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.164686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.164982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.165245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.165949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:30.166311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.167780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.206145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 487/979] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:30.207609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.207896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.208176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.223011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 493/980] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:30.223011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.230260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:30.231974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:30.319788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 488/981] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:30.320786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.354215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.396117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f1457-d589-4ada-921c-7a6eabf7dda4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.456103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.456631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.456903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.457178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.458553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] RBAC: Adding query filter params (interface=admin, service_id=3cf23395920840e5a513ae58d8fab1c0, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:30.458908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.461698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.467116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f353e40-7ce4-4cda-8015-b9e7d6e71ee0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.473743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 494/982] 10.222.0.73 () {56 vars in 1249 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/endpoints?service_id=3cf23395920840e5a513ae58d8fab1c0&interface=admin®ion_id=RegionOne => generated 193 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:30.483373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.503728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.521271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.522015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.522370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.522771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.547356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.549855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:30.574188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:30.575290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:30.576217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:30.576217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:30.576217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:30.576217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 489/983] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:30.582077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.616656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2e7f4-c67f-447b-856b-7b80235f39e0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.662415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.663048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.663385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.663656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.664320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.665686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.672063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 495/984] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:25:30.674037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.683178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.708394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d785b4-a37c-47f7-9d73-59e454e6d41e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.761920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.763530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.763840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.764097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.764844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:30.765221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.767838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.774328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:30.788790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 490/985] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:30.790663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:30.810578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86075729-50d8-4ee8-be09-36a102cb86dc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:30.861998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:30.862612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:30.862959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:30.863257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:30.863881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:30.865339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:30.882119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 496/986] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:30 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:30.946277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf8ad-bbe7-41e4-9152-9b7f4d48df8f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:31.032798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:31.033505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:31.033941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:31.038325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:31.042594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:31.048632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:31.050266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:31.066925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:31.067881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:31.069196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 491/987] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:31.074016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:31.185453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7c9e6-6aa1-4eb9-84b8-8d6d8c424da6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:31.220099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:31.220810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:31.221150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:31.221430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:31.222128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:31.222435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:31.223906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881b184-cb7e-4d89-ba29-79b3c780d6bd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:31.232512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:31.244221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 497/988] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:31.246045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:31.356417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fde2-91cd-47a9-8769-1e7f267bdd33 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:31.394397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:31.395029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:31.395305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:31.395571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:31.396294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] RBAC: Adding query filter params (role.id=ab45b3a2512744b8948b506ea9dac763, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=fbfd83920ceb42598c2f2ba9b008a94a) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:31.396599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:31.398006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e661e0-332e-4c68-8a9e-bc58222b55b8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:31.404397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 492/989] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:31 2022] GET /identity/v3/role_assignments?user.id=fbfd83920ceb42598c2f2ba9b008a94a&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=ab45b3a2512744b8948b506ea9dac763 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:32.635082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b486c6-e023-4880-af0d-c3e81b949deb None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:32.635611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b486c6-e023-4880-af0d-c3e81b949deb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:32.635611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b486c6-e023-4880-af0d-c3e81b949deb None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:32.637045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 498/990] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:32.642135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169ac11-4453-413c-894f-ad321b3c77ae None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:32.642554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169ac11-4453-413c-894f-ad321b3c77ae None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:32.642741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169ac11-4453-413c-894f-ad321b3c77ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:32.661756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b169ac11-4453-413c-894f-ad321b3c77ae None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:32.662982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b169ac11-4453-413c-894f-ad321b3c77ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:32.720522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:32.734699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 493/991] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:32.885553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c5b5-8dbb-4e0d-848e-246c23f4ccb4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:32.885755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c5b5-8dbb-4e0d-848e-246c23f4ccb4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:32.886019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c5b5-8dbb-4e0d-848e-246c23f4ccb4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:32.887553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 499/992] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:32.893330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:32.893689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:32.894063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:32.913041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:32.914240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:32.984384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 494/993] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:33.003930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8c5b5-8dbb-4e0d-848e-246c23f4ccb4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.107099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8c5b5-8dbb-4e0d-848e-246c23f4ccb4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.143153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.143703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.143982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.144253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.144831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.146184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:33.154883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:33.155910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:25:33.156915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.159006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 500/994] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:33.162243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.259227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.265939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5194363-0d1b-4fc1-9efb-0ebe9319d1e1 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.287870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.304634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.305248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.305523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.305791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.306430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:33.306768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.308128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.318379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 495/995] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:33.319310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.333861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.373722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38657d-135b-40db-bb39-3b118251c89a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.408373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.408931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.409211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.409484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.410072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.411475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.418191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cc4e3-a7f3-4ab9-b6ea-86715755d07b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-71ddd4aa-2023-4708-8f11-bdf4ed830b84 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:33.419988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:33.421008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.422002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 501/996] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:33.427295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a896a4-e7d8-499f-a34c-ea9edd556a0b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.427569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a896a4-e7d8-499f-a34c-ea9edd556a0b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.427849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a896a4-e7d8-499f-a34c-ea9edd556a0b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.429193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 502/997] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:33.429893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a896a4-e7d8-499f-a34c-ea9edd556a0b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.463005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.463734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.464141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.464518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.465302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:33.465607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.466993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761978f4-ebfb-4027-964e-c820262e3f97 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.472675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 496/998] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:33.473900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.474192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.474469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.493172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:33.494357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:33.537443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a896a4-e7d8-499f-a34c-ea9edd556a0b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.539410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.563088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 497/999] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:33.563381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.571421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.572465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.573072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.573351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.573626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.574257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:33.574559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.575954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.583037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 503/1000] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:33.586831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.677792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8fc91-42b4-4bda-92d0-45b4ef571aa6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.699626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815795e9-9904-44ca-81dd-7a6d13e0e856 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.714389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.715265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.715544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.715818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.716542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.717901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.736005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 498/1001] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:33.745057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.745688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.745990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.746249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.746944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:33.747256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.748708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.759768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 504/1002] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:33.761480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.807429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.833651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.886671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267288-6fe8-4d0b-8d4f-8b28737b7366 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.893842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:33.925017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.925629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.925905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.926175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.926906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] RBAC: Adding query filter params (interface=public, service_id=9318a32cb8be4c09b1f972723fe6c98f, region_id=RegionOne) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:33.927216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.928620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.934677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 499/1003] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/endpoints?service_id=9318a32cb8be4c09b1f972723fe6c98f&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:33.939786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:33.947861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a642-f73c-4d3f-9e0c-1a6f63e60957 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:33.985832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:33.986473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:33.986779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:33.987074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] PATH_INFO: `/v3/services/image` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:33.987646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:33.989018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:33.997733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:33.998819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:33.999675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 505/1004] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:34.004993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:34.050284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed6447-8c47-492d-be29-671269c1886f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:34.052297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:34.087274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:34.087887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:34.088173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:34.088450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:34.089066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:34.090429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:34.098392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 500/1005] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:33 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:25:34.126490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b37ed-7bf4-46f0-8bbc-a1a304a3ec48 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:34.142491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:34.167162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:34.168075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:34.168526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:34.168939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:34.169743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:34.170189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:34.171776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010e0ac-5ff7-402d-8554-96dab152f1f9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:34.178367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 506/1006] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:34.184318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:34.295810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3439-ddb4-439d-bf71-0af6bb4f856f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:34.339225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:34.339835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:34.340120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:34.340394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:34.341027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] RBAC: Adding query filter params (type=image) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:34.341333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:34.342779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21daf94-a8e3-4dab-a816-f5d5608abcbc None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:34.347771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 501/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:35.677737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8a6c-3f01-4e80-a77a-f3cf12e0349f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:35.678230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8a6c-3f01-4e80-a77a-f3cf12e0349f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:35.678230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8a6c-3f01-4e80-a77a-f3cf12e0349f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:35.679596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 507/1008] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:35.684654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcdd39-66d6-42c7-8240-a9d6f9f31fb1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:35.685047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcdd39-66d6-42c7-8240-a9d6f9f31fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:35.685204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcdd39-66d6-42c7-8240-a9d6f9f31fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:35.703529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-cdbcdd39-66d6-42c7-8240-a9d6f9f31fb1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:35.704770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cdbcdd39-66d6-42c7-8240-a9d6f9f31fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:35.728951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:35.777315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 502/1009] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:36.036579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1892d-bf4e-401f-8c45-a348ab5ca7f7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.036950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1892d-bf4e-401f-8c45-a348ab5ca7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.037139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1892d-bf4e-401f-8c45-a348ab5ca7f7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.038691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 508/1010] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:36.043792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.044331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.044618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.045055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1892d-bf4e-401f-8c45-a348ab5ca7f7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.067179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:36.068407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:36.146960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 503/1011] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:36.154173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1892d-bf4e-401f-8c45-a348ab5ca7f7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.189857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.190477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.190783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.191053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.192169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.193582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c61445-5a0c-40f4-a32a-73e10237ed31 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.195341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 509/1012] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:36.200815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.291239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:36.299195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9427-1554-4959-91db-ca29ef16cdc9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.299487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9427-1554-4959-91db-ca29ef16cdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.299768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9427-1554-4959-91db-ca29ef16cdc9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.301248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 510/1013] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:36.302207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fa4aa-2c6a-4154-9a3c-82bb3c28373a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.306262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.306575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.306879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.332054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:36.333570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:36.340566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.341174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.341458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.341737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.342394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.344037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.374708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 504/1014] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:36 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:36.379042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:36.408033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 511/1015] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:36.411134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.524250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5795-b27d-46c1-8b7e-58137c0d9d4b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.560222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.560222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.560498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.560722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.561192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.562561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:36.574066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:36.575122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:25:36.575994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:36.578415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 505/1016] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:36.585808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.678282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.689831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e5423-7c63-4504-bd83-4aee696307e1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.692228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:36.725881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.726492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.726856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.727133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.727771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:36.728146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.729558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.735234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 512/1017] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:36.741101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:36.793607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:36.803514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1e33b-f9a0-4313-a84d-a71932a2a5bf None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.840747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.841282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.841562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.841831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.842477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.843869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.862563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 506/1018] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:25:36 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:25:36.872923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab74f9-a131-409f-88a7-37f77d7e9f1b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:36.920978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:36.921594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:36.921876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:36.922197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:36.922944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:36.923264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:36.924683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6aeb3-5619-4c8e-bae9-c298fb7885f4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:36.935332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 513/1019] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 15:25:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:36.936344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:37.010715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:37.049786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12bb-131c-4e84-8164-aa712bf48939 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:37.085489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:37.086112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:37.086395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:37.086692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:37.087358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:37.088731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c3f64-12b7-44d4-bb49-305010b3ccfc None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:37.111972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 507/1020] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:38.364650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe604-3184-4af8-bd27-00ac1750de7e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:38.365156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe604-3184-4af8-bd27-00ac1750de7e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:38.365266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe604-3184-4af8-bd27-00ac1750de7e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:38.366767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 514/1021] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:38.371781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9c20-cf7f-4cb1-8d08-303f4e4528d8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:38.372199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9c20-cf7f-4cb1-8d08-303f4e4528d8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:38.372388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9c20-cf7f-4cb1-8d08-303f4e4528d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:38.391576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-eabb9c20-cf7f-4cb1-8d08-303f4e4528d8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:38.393482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-eabb9c20-cf7f-4cb1-8d08-303f4e4528d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:38.470172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 508/1022] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:38.743506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe604-3184-4af8-bd27-00ac1750de7e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:38.772440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88719fec-9d63-4b91-ae7c-8e65f0dbcf24 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:38.772729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88719fec-9d63-4b91-ae7c-8e65f0dbcf24 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:38.773187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88719fec-9d63-4b91-ae7c-8e65f0dbcf24 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:38.774619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 509/1023] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:38.779872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:38.780158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:38.780440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:38.800006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:38.801178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:38.848248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:38.856959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:38.862479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe604-3184-4af8-bd27-00ac1750de7e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:38.890938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 510/1024] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:38.902690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:38.903331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:38.903610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:38.903880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:38.904997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:38.906437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:38.913640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 515/1025] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:38.914505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.014340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99914d97-4311-4d11-afc3-803ac43b6af8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.048882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.049555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.049833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.050103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.050792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.052201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27773da6-9d94-42d2-b2f9-c936694fbc83 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.074100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 511/1026] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:25:38 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:39.153791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.154485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad337-29d9-4d48-98fb-5cb1a40f8df3 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.154889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad337-29d9-4d48-98fb-5cb1a40f8df3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.155171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad337-29d9-4d48-98fb-5cb1a40f8df3 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.156549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 512/1027] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:39.165797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.166138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.166478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.185439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:39.186693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:39.222079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:39.273146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 513/1028] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:39.290314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79574b24-b9ee-4812-bd17-d753ed52bfce None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.328017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.328654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.328933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.329818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.330406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.331928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:39.340813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:39.341848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:39.343133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 516/1029] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:39.348186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.452099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee64dea-db88-4afd-a4e3-1478c571e01a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.487016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.487615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.487891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.488162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.488784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] RBAC: Adding query filter params (name=image) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:39.489084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.490438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.495411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 514/1030] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:39.501401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.534156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:39.552556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.583215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:39.608366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec39ee-fbba-49f2-9d40-cf8e3731efd1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.646004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.646826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.647112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.647385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.648023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] RBAC: Adding query filter params (type=image) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:39.648328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.649697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.659904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 517/1031] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:39.660889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.668534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9a0a-2755-496b-b117-9e665168f03f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.704398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.705107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.705384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.705650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.715644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:39.721093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:39.722034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:39.722909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 515/1032] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:39.727940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.771703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:39.779547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb97ad-ecfd-4116-9edf-fc2035ab898e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.821471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.821995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.822280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.822550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.823229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:39.823343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] RBAC: Adding query filter params (interface=public, service_id=ed88ae946a0c4148b4baaf00bad122de, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:39.823565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.824946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.831285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 518/1033] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/endpoints?service_id=ed88ae946a0c4148b4baaf00bad122de&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:39.836797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.842212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35595-f023-46dc-a36a-db573bcb2a4a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.879318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.880181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.880465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.880735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.881404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:39.881706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.883112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:39.891490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 516/1034] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:39.899400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:39.952291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c349f-04db-4a12-9e05-def0ea29a191 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:39.988481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:39.989153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:39.989440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:39.989721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:39.990331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:39.991741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.006681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 519/1035] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:25:40.008157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46186e-e8c9-4d0d-aa55-7394aa76e258 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:40.045184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:40.045826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:40.046126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:40.046397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:40.058601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:40.060088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:40.075892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:40.076894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:40.078549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:40.078549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:40.078549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:40.078549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 517/1036] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:40.083696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:40.112575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:40.197265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a3c0-f23a-4710-90a7-976decad7132 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:40.234095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:40.234732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:40.235012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:40.235346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:40.236005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:40.236500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:40.237867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.250090 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 520/1037] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:40.255959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:40.258948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:40.367461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02040a-9d4d-4399-b531-037af1aa2889 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:40.401753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:40.402719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:40.403206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:40.403681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:40.412015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:40.413769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:40.426122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:40.427200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:40.428646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 518/1038] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:25:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:40.433720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:40.524007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:40.542268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de51d26-63d2-4751-9a82-cd33190de651 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:40.577212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:40.577815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:40.578091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:40.578360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:40.579138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:40.579471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:40.580890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.599220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 521/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:25:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:40.604638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:40.667587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:40.707894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbfb58-8b46-4208-a417-7fd3c6d4b94f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:40.741412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:40.742011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:40.742289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:40.742556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:40.809165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:40.810605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf26d6-1dbb-45c6-9e76-c0013732b03c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:40.872751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:40.918870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 519/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:40.920673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.025233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcede1a7-6010-4826-b0cc-b4e13561e98b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.059097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.059735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.059906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4c2a-59da-497d-96f3-e58f63e590b7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.060037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.060201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4c2a-59da-497d-96f3-e58f63e590b7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.060305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.060453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4c2a-59da-497d-96f3-e58f63e590b7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.061906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 520/1041] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:41.067959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.068248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.068524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.071601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.073034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.074792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.087880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:41.089086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:41.090025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:41.090982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:41.091871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:41.091871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:41.091871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:41.091871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 522/1042] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:41.096953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.174829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 521/1043] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:41.201292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34108edb-911c-4b47-a093-b1aefc6fa933 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.234451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.235607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.235884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.236157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.236826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:41.237134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.238494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.250213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 523/1044] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:41.256408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.355103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.366275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e7744-613a-4807-945b-8646fb3fddb2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.401402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.402030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.402308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.402581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.409446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.410910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:41.422049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:41.423538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:41.429356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 522/1045] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:41.432199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.432356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.494682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.544214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2f60e-d584-4a40-85c7-4c14bd0679cd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.544954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f0a4f-4031-407d-9c3c-f71a963010b3 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.581515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.582144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.582414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.582730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.583264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.584082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.584578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.585063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.585967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:41.586473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.588128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.593631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:41.598890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:41.599825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:41.600694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 523/1046] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:41.607637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 524/1047] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:41.607778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.608734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.626153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.717248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43705a2-1cc4-427a-a0e9-f51a01fcf12a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.721340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee5f22-e5c7-4951-b815-9d54ed220997 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.736145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.754314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.754977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.755261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.755539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.756200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:41.756508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.757873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.758154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.758786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.759167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.759333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.760043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] RBAC: Adding query filter params (role.id=ab45b3a2512744b8948b506ea9dac763, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:41.760553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.761778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.768024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 524/1048] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:41.773921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.779025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 525/1049] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=ab45b3a2512744b8948b506ea9dac763 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:41.862345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:41.891589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3ff3d-7cc2-43bf-bf5a-959df15f6050 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:41.929194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:41.929827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.930105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.930378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] PATH_INFO: `/v3/users/37efcc3f3944417ab978d6c285c32ddc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.944372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] RBAC: Authorizing `identity:get_user(user_id=37efcc3f3944417ab978d6c285c32ddc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:41.945806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd17851-e917-4ac2-a980-ba288f3845c2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:41.952235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 525/1050] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/users/37efcc3f3944417ab978d6c285c32ddc => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:41.954400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:41.961906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609dab7-4fd2-41e0-8434-3d7734f2f325 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.962190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609dab7-4fd2-41e0-8434-3d7734f2f325 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.962466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609dab7-4fd2-41e0-8434-3d7734f2f325 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.963938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 526/1051] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:41.969342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:41.969731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:41.970077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:41.988458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:41.989660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:42.060090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 527/1052] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:42.073139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3270da-b845-4a2f-bf9a-f7c08a2c9ab4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.107385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.107985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.108527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.108799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.163739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:42.191566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.193041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.298817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 526/1053] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:42.302424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.346311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.380023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:42.423485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91f6f4-defa-4519-942e-d31df5d34a79 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.426777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:42.452537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20eb2c-f116-4f4e-95f2-96510554afb6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.462703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.463560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.464314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.464761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.465899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.467603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.469830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 528/1054] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:25:42 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:42.475903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.489490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.490119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.490399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.490696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] PATH_INFO: `/v3/services/image` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.491279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.492650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:42.502079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:42.503141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:25:42.504008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:42.506794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 527/1055] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 15:25:42 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:42.508957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.589957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7a1c2-9be4-4870-965d-4ca851475b80 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.615837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a3404-be70-4230-8a9b-37910af24adc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.621898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:42.629167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.629764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.630039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.630309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.631152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=023fffa26aff44f5a96df141691bf524, user.id=37efcc3f3944417ab978d6c285c32ddc) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:42.631464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.632877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.646689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 529/1056] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:42 2022] GET /identity/v3/role_assignments?user.id=37efcc3f3944417ab978d6c285c32ddc&scope.project.id=023fffa26aff44f5a96df141691bf524&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:42.660605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.661229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.661508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.661783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.662433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] RBAC: Adding query filter params (name=image) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:42.662770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.664167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.669899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 528/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:42.676804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.790958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e8f0f-067d-4b9a-8bbe-8a5420d2bd15 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.825685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.826302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.826609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.826898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.827545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:42.827848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.829770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd4282-18b7-4852-b723-cbbaaf2a6cd5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:42.835102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 530/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:42.841244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:42.852373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:42.947465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f548-4da6-4a25-9cbf-a6983fd3641b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:42.981916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:42.982518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:42.982829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:42.983368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:42.984026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:42.985429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d84fb1-feba-4c1f-9331-5dde864e4ea7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:43.014666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 529/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:25:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:43.695745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505139-a591-4a9b-afe5-3efc3db490b9 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:43.696325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505139-a591-4a9b-afe5-3efc3db490b9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:43.696325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505139-a591-4a9b-afe5-3efc3db490b9 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:43.697724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 531/1060] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:43.702602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:43.703067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:43.703280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:43.722138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:43.723407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:43.780230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:43.799411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 530/1061] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:44.065272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505139-a591-4a9b-afe5-3efc3db490b9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.130093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:44.177192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505139-a591-4a9b-afe5-3efc3db490b9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:44.211990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:44.212618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.212900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.213168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.224514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:44.229233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:44.230244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:44.231501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:44.235162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 532/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:44.237862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.340211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e2599-a0ce-49ee-91cf-e761023be2c8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:44.376101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:44.376706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.376985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.377258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.377978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:44.378287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:44.379674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:44.393255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 531/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:44.394457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.496873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfa86-6e8c-454f-8b23-b782a103080e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:44.514111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:44.532743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:44.533377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.533659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.533935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.544732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:44.546177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b84fa25c-d45b-4668-b235-a21b06b6d9ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:44.560480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:44.561567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:44.567709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:44.567709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:44.567709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:44.567709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 533/1064] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:44.569609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.575660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee679e4-27c9-449b-a9eb-a9b001d853f0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.575910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee679e4-27c9-449b-a9eb-a9b001d853f0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.576166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee679e4-27c9-449b-a9eb-a9b001d853f0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.577628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 534/1065] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:25:44.583558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.583848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.584134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.601969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:44.603180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:44.627888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:44.676222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 535/1066] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:25:44.682525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895c88f-d17c-4c92-a995-790a4125c0a0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:44.718571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:44.719237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.721847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.721847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.721847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:44.721847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:44.724080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:44.737064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 532/1067] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:44.742464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.846000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:44.849498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f3383-249e-481f-a6c1-1dacc64d4851 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:44.885841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:44.886471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:44.886801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:44.887076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:44.894941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:44.896385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:44.907943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:44.908917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:44.914632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 536/1068] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:44.915282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:44.944578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.018414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34efc3-08aa-489b-9621-93919532abe7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.028815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.046150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc516c1-cf0c-4389-9dc7-f34e89bf415a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.053078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.053684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.053963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.054238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.054999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:45.055573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.056733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27d5bc-58b5-4dfe-ba89-f68603f69ac0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.073549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 533/1069] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:45.073549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e692a-7346-4e71-93d9-cd9c1a58c92b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.073549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e692a-7346-4e71-93d9-cd9c1a58c92b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.073549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e692a-7346-4e71-93d9-cd9c1a58c92b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.073549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 534/1070] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:45.079726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e692a-7346-4e71-93d9-cd9c1a58c92b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.084475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.085107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.085390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.085667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.098092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b21db459-b27c-4f97-8061-4571b03cf070 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:45.102800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:45.103765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:45.104635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 537/1071] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:45.105357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.105434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.105728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.123626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:45.124858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:45.138626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.185319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e692a-7346-4e71-93d9-cd9c1a58c92b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.198060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 538/1072] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:45.198932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.235525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.236158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.236963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.237250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.248312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.250309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:45.265020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:45.266598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:45.267997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:45.267997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:45.267997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:45.267997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 535/1073] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:45.272529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.294901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.309909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ae85a-7f2d-4e0f-9ab0-27f5201c0259 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.351182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.351718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.351997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.352269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.352932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:45.353234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.354594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.364220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 539/1074] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:45.369757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.385273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.388410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fc797-6a9a-4482-9c37-a0e162f17485 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.427874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.428586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.428866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.429133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.429778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:45.430082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.431602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.448374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 536/1075] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:45.449660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.488231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b4f50-e0c4-46b5-9d81-8edc0194a68a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.522650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.523298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.523577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.523849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] PATH_INFO: `/v3/users/37efcc3f3944417ab978d6c285c32ddc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.524978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] RBAC: Authorizing `identity:get_user(user_id=37efcc3f3944417ab978d6c285c32ddc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.526391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.528017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 540/1076] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/users/37efcc3f3944417ab978d6c285c32ddc => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:45.531337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.531732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.555083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec333e2-070f-4268-8859-c00c97f622f5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.602899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.603452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.603730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.604008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.611302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.612760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.617505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:45.624867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:45.626906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 537/1077] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:45.628018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.642332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fa945-9497-4325-958c-c3c88e9d51ed None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.678632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.679541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.679820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.680091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.745994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a62590-3bdb-474a-b6e3-b16292468643 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.748964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.751222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.780142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.780852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.781224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.781501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.782817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.784256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.785887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 538/1078] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:45.786658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.798991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.850077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:45.872892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 541/1079] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:45.873610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.907308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809adad-5a7d-47ce-8cdf-71fa3662bf23 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:45.948404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:45.949009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:45.949292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:45.949562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:45.950251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:45.950554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:45.952221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:45.966797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 539/1080] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:45.967429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:45.987968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7aa90-8140-4ab5-a15b-87b289ec6e4f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:46.028505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:46.029205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:46.029485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:46.029760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524/users/37efcc3f3944417ab978d6c285c32ddc/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:46.032768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] RBAC: Authorizing `identity:create_grant(project_id=023fffa26aff44f5a96df141691bf524, user_id=37efcc3f3944417ab978d6c285c32ddc, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:46.034242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:46.047508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:46.050499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 542/1081] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:45 2022] PUT /identity/v3/projects/023fffa26aff44f5a96df141691bf524/users/37efcc3f3944417ab978d6c285c32ddc/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:46.057902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:46.099976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc305ed-545c-47da-9ed4-d659fbbb6dc0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:46.114553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:46.146992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:46.147960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:46.148546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:46.149006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:46.162400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:46.164139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:46.184152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:46.185194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:46.186078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:46.186078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:46.186078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:46.186078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 540/1082] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:46.192342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:46.193928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfa20b-cb58-424a-9be2-052fdf171132 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:46.255469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:46.256765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:46.257440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:46.257928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/ab45b3a2512744b8948b506ea9dac763` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:46.261835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=8eb093eda2a14771aef59b6269ebe789, role_id=ab45b3a2512744b8948b506ea9dac763)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:46.265052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c0713-695a-4bc6-890e-dbb3d210ed65 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:46.287075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 543/1083] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:46 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/8eb093eda2a14771aef59b6269ebe789/roles/ab45b3a2512744b8948b506ea9dac763 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:46.312246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df63567-3339-433e-beeb-f14bfcfe4ce0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:46.343547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:46.361250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:46.361873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:46.362156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:46.362431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:46.363134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:46.363443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:46.364841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fd747-b9c1-4703-942e-28d52f53f008 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:46.379656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 541/1084] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:48.012678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5be5eb-ec83-4a66-b792-63ae05d78965 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.013200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5be5eb-ec83-4a66-b792-63ae05d78965 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.013321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5be5eb-ec83-4a66-b792-63ae05d78965 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.014607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 544/1085] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:48.020106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3dab-6e9c-43ad-889a-11b2c2dd0306 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.020505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3dab-6e9c-43ad-889a-11b2c2dd0306 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.020766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3dab-6e9c-43ad-889a-11b2c2dd0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.038564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ff0e3dab-6e9c-43ad-889a-11b2c2dd0306 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:48.039770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ff0e3dab-6e9c-43ad-889a-11b2c2dd0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:48.105302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 542/1086] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:48.308998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662742da-8739-4e83-abbb-3577b6c7fc5a None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.309178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662742da-8739-4e83-abbb-3577b6c7fc5a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.309459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662742da-8739-4e83-abbb-3577b6c7fc5a None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.310843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 545/1087] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:48.316267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.316552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.316833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.334325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:48.335520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:48.399646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 543/1088] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:48.403543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662742da-8739-4e83-abbb-3577b6c7fc5a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.501213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:48.507660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662742da-8739-4e83-abbb-3577b6c7fc5a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.544139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:48.544761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.545037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.545303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.556006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:48.560716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:48.561817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:48.563369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 546/1089] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:48.567984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.609891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:48.666791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac9821-1f15-475e-a7c7-941edbcd11b7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.669785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.700186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:48.700786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.701074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.701344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.702602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:48.702944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:48.704322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:48.714491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 544/1090] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:48.718103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.770059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7a2d-81af-4436-8d72-f57f4d5dad90 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.806345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:48.807239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.807522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.807787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.820257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:25:48.825189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dd29-c918-4937-9d04-1a5e7d510183 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:25:48.825968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:48.827961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:48.829370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 547/1091] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:48.831418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:48.833856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.856090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:48.864036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:48.864669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.864951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.865229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] PATH_INFO: `/v3/users/37efcc3f3944417ab978d6c285c32ddc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.866359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] RBAC: Authorizing `identity:get_user(user_id=37efcc3f3944417ab978d6c285c32ddc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:48.867816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:48.869401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 545/1092] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/users/37efcc3f3944417ab978d6c285c32ddc => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:48.875919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:48.940188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a8f68-e259-4d4c-99df-ac64205dc1ea None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.972793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:48.973390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:48.973667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:48.973945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:48.974195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8abe-060e-4640-beed-e77eb7f53030 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:48.974604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:48.974935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:48.976285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:48.984958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 548/1093] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:48.990502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.010368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.011001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.011283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.011557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.052488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.073409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.081216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.082708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.091884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee695-6500-4495-8ad9-81eb070709b8 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.127323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.127948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.128224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.128496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.138974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.141197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:49.158402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:49.159592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:49.160695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:49.160695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:49.160695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:49.162788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 549/1094] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:25:49.165371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.189237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 546/1095] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 15:25:49.190414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.258140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.265019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb164d-88ec-4a59-89a6-8c0dd75692e0 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.289364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747087c6-d701-4cf2-b11e-322a94c552c0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.298944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.299549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.299828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.300102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.300748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:49.301051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.302410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.304252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.321251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 550/1096] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:49.322618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.329116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.329681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.329963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.330236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.331456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.332880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.334525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 547/1097] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:49.340380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.428326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d090-3279-4f52-9e88-801953dbca59 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.439627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae8040-af1f-46c6-94c5-982f937e5c98 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.461877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.462500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.462814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.463096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.470557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.472050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.473922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.474521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.474827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.475102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.475816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=023fffa26aff44f5a96df141691bf524, user.id=37efcc3f3944417ab978d6c285c32ddc) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:49.476121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.477515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.483981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 548/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/role_assignments?user.id=37efcc3f3944417ab978d6c285c32ddc&scope.project.id=023fffa26aff44f5a96df141691bf524&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:25:49.484280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:49.485285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:25:49.486257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 551/1099] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:49.490772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.493716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.600663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14269886-9fd8-46fb-8dd2-e3f02530cc76 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.638437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.639617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.639934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.640426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.641279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:49.641610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.643074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.661136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 549/1100] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:49.667084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:49.722718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.767311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cdb5b-ad8d-4b6f-9e38-533ae2669498 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:49.800966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:49.801562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:49.801837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:49.802106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:49.869109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:49.870455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:49.925415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:49.970148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 552/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:49.976041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:50.037738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:50.078239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779702ca-1f20-4324-b48e-63bff13ff83f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:50.112263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:50.112900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:50.113183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:50.113460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:50.123352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:50.124849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:25:50.138966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:50.140446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:25:50.141355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:25:50.141355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:25:50.141355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:50.141355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 550/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:50.146599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:50.245863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c596f08-563c-4adc-a32a-2904cccc3ece None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:50.278904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:50.279495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:50.279909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:50.280176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:50.280809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:50.281109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:50.282457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:50.293719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 553/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:50.300886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:50.401283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d44549-0ad5-45dc-8afb-201abea0c533 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:50.413705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:50.434897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:50.435519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:50.435792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:50.436760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:50.443143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:50.444573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:25:50.455334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:25:50.456376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:25:50.457927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 551/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:25:50.462254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:50.481264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:50.566607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032da914-c540-4a45-8416-9ecffeafcf91 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:50.600914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:50.601526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:50.601805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:50.602079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:50.602811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:50.603120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:50.604550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff34cf8-e99e-428e-a57f-54dec20778df None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:50.617555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 554/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:50.624026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:50.722059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7869aee-1790-466e-b527-84dead8d5876 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:50.754569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:50.755185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:50.755464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:50.755735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:50.756452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] RBAC: Adding query filter params (role.id=ab45b3a2512744b8948b506ea9dac763, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=8eb093eda2a14771aef59b6269ebe789) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:50.756760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:50.758179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725037b-f1d0-4b45-97ce-1b1f8b167231 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:50.763828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 552/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:50 2022] GET /identity/v3/role_assignments?user.id=8eb093eda2a14771aef59b6269ebe789&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=ab45b3a2512744b8948b506ea9dac763 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:51.438616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53b4c0-c02d-488e-86c0-c7d2f9fd02b2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:51.439136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53b4c0-c02d-488e-86c0-c7d2f9fd02b2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:51.439251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53b4c0-c02d-488e-86c0-c7d2f9fd02b2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:51.440530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 555/1107] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:51.446059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:51.446490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:51.446592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:51.464829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:51.466020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:51.496278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:51.535697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 553/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:51.804630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53b4c0-c02d-488e-86c0-c7d2f9fd02b2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:51.879691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:51.905955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53b4c0-c02d-488e-86c0-c7d2f9fd02b2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:51.938578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:51.939215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:51.939490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:51.939758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:51.940846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:51.942248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6ee7b-ec8d-4063-930e-ddf4a1c71320 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:51.943826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 556/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:25:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:51.949549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:52.046297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be22ed-32ea-4c77-af4d-95a572f6034f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:52.079790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:52.080322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:52.080601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:52.080875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:52.081510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:52.082958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f500b-e7c6-4bcb-b271-31c6aadb7ca5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:52.103766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 554/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:25:51 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:25:54.042460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c320e-4c74-403c-9332-2d7b18bf437c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:54.043035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c320e-4c74-403c-9332-2d7b18bf437c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:54.043140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c320e-4c74-403c-9332-2d7b18bf437c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:54.044535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 557/1111] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:54.049910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:54.050338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:54.050474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:54.069098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:54.070272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:54.086902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:54.136442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 555/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:54.416001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c320e-4c74-403c-9332-2d7b18bf437c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:54.515383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c320e-4c74-403c-9332-2d7b18bf437c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:54.548783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:54.549395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:54.549677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:54.549953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] PATH_INFO: `/v3/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:54.551361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] RBAC: Authorizing `identity:get_role(role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:54.552720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:54.554278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 558/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:54 2022] GET /identity/v3/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:54.560171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:54.657457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854537f-a8de-4ad4-a014-6cbcb84ab8ab None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:54.689495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:54.690118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:54.690396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:54.690693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] PATH_INFO: `/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:54.704131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c4f05ce7fe4a9fbbc3131c04b106a9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:54.705558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:54.707187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 556/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:54 2022] GET /identity/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:54.712952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:54.725674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:54.812977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9f084-0fe4-4177-93e0-e8fe78dcd0d7 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:54.848346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:54.848942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:54.849220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:54.849499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:54.915015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:54.916378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:54.927380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:55.012212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 559/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:25:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:25:55.018207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:55.031098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:55.116152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db062828-67fc-44cc-b86c-d00e7f5712f0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:55.149430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:55.150047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:55.150323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:55.150593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:55.151725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:55.153138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a069ed-bd98-489a-89f6-0a587cc4a378 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:55.155308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 557/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:25:55 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:55.161548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:55.258096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334a23b-31eb-4879-8560-bc12b38620d4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:55.268609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:55.292132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:55.292725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:55.293004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:55.293279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:55.294001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=023fffa26aff44f5a96df141691bf524, user.id=c6c4f05ce7fe4a9fbbc3131c04b106a9) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:25:55.294880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:55.296303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712c585-8ca2-4864-834c-e65aba74f7f3 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:55.301478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 560/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:25:55 2022] GET /identity/v3/role_assignments?user.id=c6c4f05ce7fe4a9fbbc3131c04b106a9&scope.project.id=023fffa26aff44f5a96df141691bf524&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:57.177441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2035f-21af-4f30-867b-b3043235a57b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:57.177988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2035f-21af-4f30-867b-b3043235a57b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:57.177988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2035f-21af-4f30-867b-b3043235a57b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:57.179359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 558/1118] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:25:57.184971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:57.185414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:57.185682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:57.203837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:25:57.205025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:25:57.270972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 561/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:25:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:25:57.539974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2035f-21af-4f30-867b-b3043235a57b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:57.619641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:57.644225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2035f-21af-4f30-867b-b3043235a57b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:57.679393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:57.680169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:57.680449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:57.680758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] PATH_INFO: `/v3/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:57.682238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] RBAC: Authorizing `identity:get_role(role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:57.683657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:57.685774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 559/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:57 2022] GET /identity/v3/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:57.691660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:57.792803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a3bb1-e52e-4fb9-8c46-29a4615024a8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:57.795944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:57.830579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:57.831258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:57.831538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:57.831813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] PATH_INFO: `/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:57.832944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c4f05ce7fe4a9fbbc3131c04b106a9)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:57.834371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:57.836295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 562/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:25:57 2022] GET /identity/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:57.841855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:57.939986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62c28-f982-4969-890e-d6193e20c98a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:57.972561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:57.973184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:57.973468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:57.973741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:57.974878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:57.976300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fbaf7-3aaf-4d67-84e3-cb44885e4f8c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:57.977916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 560/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:25:57 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:25:57.984874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:25:58.081615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6d528-9cbb-435d-8a5c-96fabe862fe9 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:25:58.115385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:25:58.116021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:58.116301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:58.116578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524/users/c6c4f05ce7fe4a9fbbc3131c04b106a9/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:58.118712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] RBAC: Authorizing `identity:create_grant(project_id=023fffa26aff44f5a96df141691bf524, user_id=c6c4f05ce7fe4a9fbbc3131c04b106a9, role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:25:58.120135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3608b5-2ff8-4bb7-a5b0-031fb67b5e62 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:25:58.132519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 563/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:25:57 2022] PUT /identity/v3/projects/023fffa26aff44f5a96df141691bf524/users/c6c4f05ce7fe4a9fbbc3131c04b106a9/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:25:59.997563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b061d-6bf8-4107-a672-662c5ab9e2d5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:25:59.998091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b061d-6bf8-4107-a672-662c5ab9e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:25:59.998091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b061d-6bf8-4107-a672-662c5ab9e2d5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:25:59.999479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 561/1124] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:00.004642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:00.005028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:00.005198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:00.024328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:00.025510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:00.051176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:00.092546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 564/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:00.366024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b061d-6bf8-4107-a672-662c5ab9e2d5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:00.382037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:00.472175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b061d-6bf8-4107-a672-662c5ab9e2d5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:00.506085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:00.506730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:00.507014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:00.507291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] PATH_INFO: `/v3/roles/2d09b9c3e152493b8b063613a24f40a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:00.508678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] RBAC: Authorizing `identity:get_role(role_id=2d09b9c3e152493b8b063613a24f40a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:00.510040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:00.511390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 562/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:00 2022] GET /identity/v3/roles/2d09b9c3e152493b8b063613a24f40a3 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:00.517732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:00.612822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7cc3a-0b77-4e80-ae66-dd3a5fb1039d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:00.646053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:00.646680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:00.646960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:00.647232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] PATH_INFO: `/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:00.648322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] RBAC: Authorizing `identity:get_user(user_id=c6c4f05ce7fe4a9fbbc3131c04b106a9)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:00.649728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:00.651295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 565/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:00 2022] GET /identity/v3/users/c6c4f05ce7fe4a9fbbc3131c04b106a9 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:00.656989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:00.737511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:00.760403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41815985-77a7-43e8-ad38-3c04982e9348 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:00.792763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:00.794420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:00.794723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:00.794994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:00.862691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:00.864186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:00.941123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:00.960205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 563/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:00.966725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:00.996135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:01.064233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc7a0b-1683-4f76-bed4-8acd07c9bf7b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:01.097226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:01.097850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:01.098128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:01.098404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] PATH_INFO: `/v3/projects/023fffa26aff44f5a96df141691bf524` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:01.099540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] RBAC: Authorizing `identity:get_project(project_id=023fffa26aff44f5a96df141691bf524)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:01.100960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489283f-7013-4ea6-bfa7-f2a399bc5d86 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:01.102578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 566/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:00 2022] GET /identity/v3/projects/023fffa26aff44f5a96df141691bf524 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:01.109153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:01.205249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f247e4-43ef-43b7-b8b9-f4c64183cd58 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:01.237784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:01.238369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:01.238671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:01.238943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:01.239658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] RBAC: Adding query filter params (role.id=2d09b9c3e152493b8b063613a24f40a3, scope.project.id=023fffa26aff44f5a96df141691bf524, user.id=c6c4f05ce7fe4a9fbbc3131c04b106a9) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:01.239966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:01.241380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e6ad9-e13e-4357-a2db-076b90710c39 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:01.247727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 564/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:26:01 2022] GET /identity/v3/role_assignments?user.id=c6c4f05ce7fe4a9fbbc3131c04b106a9&scope.project.id=023fffa26aff44f5a96df141691bf524&role.id=2d09b9c3e152493b8b063613a24f40a3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:03.160699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc03db8-523d-46c3-8d4f-04e5f86406e6 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:03.161235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc03db8-523d-46c3-8d4f-04e5f86406e6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:03.161235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc03db8-523d-46c3-8d4f-04e5f86406e6 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:03.162573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 567/1131] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:03.167808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:03.168227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:03.168412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:03.185878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:03.187072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:03.227748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:03.252824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 565/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:03.522311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc03db8-523d-46c3-8d4f-04e5f86406e6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:03.618710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:03.624349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc03db8-523d-46c3-8d4f-04e5f86406e6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:03.658488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:03.659125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:03.659400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:03.659668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:03.660778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:03.662200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbf21b-944a-4df0-affb-a77a1e05536a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:03.663804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 568/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:26:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:03.670527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:03.766885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b6b97-9f1c-423c-b544-e21deb27eda1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:03.799986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:03.800517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:03.800797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:03.801072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:03.801707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:03.803165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cc1ec-d67e-4e57-a323-fee4c573cae1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:03.829817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 566/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:26:03 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:05.775285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddce8f-3dcd-4229-84ec-df3e10482da8 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:05.775818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddce8f-3dcd-4229-84ec-df3e10482da8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:05.775818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddce8f-3dcd-4229-84ec-df3e10482da8 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:05.777212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 569/1135] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:05.782507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e957-d832-4628-b91d-40064540faf4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:05.782928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e957-d832-4628-b91d-40064540faf4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:05.783106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e957-d832-4628-b91d-40064540faf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:05.801688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9612e957-d832-4628-b91d-40064540faf4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:05.802903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9612e957-d832-4628-b91d-40064540faf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:05.826473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:05.869946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 567/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:06.143487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddce8f-3dcd-4229-84ec-df3e10482da8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:06.243660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddce8f-3dcd-4229-84ec-df3e10482da8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:06.277215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:06.277829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:06.278108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:06.278381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:06.279527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:06.280948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f737601-0d12-450b-a114-ecce9a4d3039 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:06.282515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 570/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:26:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:06.288236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612e957-d832-4628-b91d-40064540faf4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:06.384375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612e957-d832-4628-b91d-40064540faf4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:06.417875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:06.418497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:06.418808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:06.419088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:06.419843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:06.422184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ed0a0-9870-4d90-ad20-86e9990d6eea None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:06.445121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 568/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:26:06 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:08.393280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555852a-bc02-4d9d-81fa-9e31a1b186da None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:08.393819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555852a-bc02-4d9d-81fa-9e31a1b186da None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:08.393819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555852a-bc02-4d9d-81fa-9e31a1b186da None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:08.395203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 571/1139] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:08.400514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054801-c952-4113-a98a-d0311b144371 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:08.400917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054801-c952-4113-a98a-d0311b144371 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:08.401076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054801-c952-4113-a98a-d0311b144371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:08.419154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-36054801-c952-4113-a98a-d0311b144371 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:08.420335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-36054801-c952-4113-a98a-d0311b144371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:08.429252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:08.493614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 569/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:08.762807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555852a-bc02-4d9d-81fa-9e31a1b186da None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:08.797831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:08.866635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555852a-bc02-4d9d-81fa-9e31a1b186da None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:08.899419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:08.900032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:08.900305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:08.900573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:08.910920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:08.915526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:08.916641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:08.917715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 572/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:08.922888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054801-c952-4113-a98a-d0311b144371 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:09.019378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054801-c952-4113-a98a-d0311b144371 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:09.053170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:09.053764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:09.054043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:09.054316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:09.055002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:09.055303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:09.056675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:09.064525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 570/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:09.070253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:09.164076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:09.171299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adfaf7-55e5-4951-ad6a-2c030f704ca0 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:09.204934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:09.205555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:09.205838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:09.206115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] PATH_INFO: `/v3/users/12da3c368565412da7f5c78641dc357e` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:09.219456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] RBAC: Authorizing `identity:get_user(user_id=12da3c368565412da7f5c78641dc357e)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:09.220897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:09.222494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 573/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:09 2022] GET /identity/v3/users/12da3c368565412da7f5c78641dc357e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:09.228422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:09.237761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:09.327520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0661ed-0298-4ac1-af17-abeaff59cf4e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:09.360452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:09.361053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:09.361333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:09.361614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:09.426762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:09.428221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:09.436245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:09.525472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 571/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:09.531931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:09.629705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3303a3-3c21-44d5-8de7-d55c2bda1540 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:09.662570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:09.663218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:09.663497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:09.663769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] PATH_INFO: `/v3/projects/8d3ce521ae414da8896979c95813e115` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:09.664882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] RBAC: Authorizing `identity:get_project(project_id=8d3ce521ae414da8896979c95813e115)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:09.666309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e2a8b-ba96-4702-a456-80f423656b16 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:09.668607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 574/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:09 2022] GET /identity/v3/projects/8d3ce521ae414da8896979c95813e115 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:09.674871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:09.770674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718efcf6-a1f7-4fee-8060-f03e13a94207 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:09.778552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:09.804968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:09.805560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:09.805842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:09.806117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:09.806875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=8d3ce521ae414da8896979c95813e115, user.id=12da3c368565412da7f5c78641dc357e) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:09.807188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:09.808705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12da7a-3f30-4702-b064-3477c74e9101 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:09.813925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 572/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:26:09 2022] GET /identity/v3/role_assignments?user.id=12da3c368565412da7f5c78641dc357e&scope.project.id=8d3ce521ae414da8896979c95813e115&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:11.668078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fca5e-76ef-44f3-ada5-05562849c5a4 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:11.668581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fca5e-76ef-44f3-ada5-05562849c5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:11.668581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fca5e-76ef-44f3-ada5-05562849c5a4 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:11.669906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 575/1147] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:11.674931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:11.675361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:11.675493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:11.693643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:11.694860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:11.765170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 573/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:12.039729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fca5e-76ef-44f3-ada5-05562849c5a4 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:12.062704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:12.140653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fca5e-76ef-44f3-ada5-05562849c5a4 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:12.173209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:12.173818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:12.174094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:12.174364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:12.184983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:12.189620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:12.190729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:12.191913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 576/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:26:12.196750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:12.292521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:12.299817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f9dd-6873-4f9b-8057-afe409e2bcb6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:12.333387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:12.333998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:12.334280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:12.334553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:12.335253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:12.335559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:12.338765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:12.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 574/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:12.358371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:12.437606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:12.456421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029d9c-871e-4e65-89f6-86e014a59f59 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:12.490141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:12.490781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:12.491067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:12.491342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] PATH_INFO: `/v3/users/12da3c368565412da7f5c78641dc357e` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:12.492458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] RBAC: Authorizing `identity:get_user(user_id=12da3c368565412da7f5c78641dc357e)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:12.493886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:12.496121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 577/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:12 2022] GET /identity/v3/users/12da3c368565412da7f5c78641dc357e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:12.501985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:12.604332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca848c-32dd-419d-9e4f-31a787e1514e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:12.636555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:12.637169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:12.637449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:12.637724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] PATH_INFO: `/v3/projects/8d3ce521ae414da8896979c95813e115` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:12.638859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] RBAC: Authorizing `identity:get_project(project_id=8d3ce521ae414da8896979c95813e115)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:12.640280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e3bd0-4b28-43b5-90ec-0fd826dc4f86 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:12.641902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 575/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:12 2022] GET /identity/v3/projects/8d3ce521ae414da8896979c95813e115 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:12.647967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:12.744659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995752d8-acd7-466b-9077-99a74c923703 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:12.777912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:12.778556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:12.778929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:12.779203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] PATH_INFO: `/v3/projects/8d3ce521ae414da8896979c95813e115/users/12da3c368565412da7f5c78641dc357e/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:12.781076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d3ce521ae414da8896979c95813e115, user_id=12da3c368565412da7f5c78641dc357e, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:12.782492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193539-a86f-4390-bdc7-78121acf9306 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:12.795834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 578/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:26:12 2022] PUT /identity/v3/projects/8d3ce521ae414da8896979c95813e115/users/12da3c368565412da7f5c78641dc357e/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:14.685183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee494057-a09c-44f0-a109-cf9a5bfe17be None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:14.685782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee494057-a09c-44f0-a109-cf9a5bfe17be None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:14.685782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee494057-a09c-44f0-a109-cf9a5bfe17be None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:14.687013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 576/1154] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:14.692111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:14.692529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:14.692668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:14.711313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:14.712499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:14.715188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:14.782212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 579/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:15.053861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee494057-a09c-44f0-a109-cf9a5bfe17be None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:15.084721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:15.154726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee494057-a09c-44f0-a109-cf9a5bfe17be None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:15.187868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:15.188483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:15.188762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:15.189033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:15.199325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:15.203994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:15.205159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:26:15.206440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 577/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:15.211167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:15.307786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f46be-5338-43f6-b41c-8ba6a9a7ef26 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:15.341356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:15.341956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:15.342236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:15.342509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:15.343209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:15.343651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:15.345014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:15.349878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:15.354292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 580/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:15.360111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:15.452397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:15.462192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9544a78-571d-4e15-8612-1d3cd6a33598 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:15.495720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:15.496333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:15.496615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:15.496891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] PATH_INFO: `/v3/users/12da3c368565412da7f5c78641dc357e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:15.498005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] RBAC: Authorizing `identity:get_user(user_id=12da3c368565412da7f5c78641dc357e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:15.499459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:15.501024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 578/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/users/12da3c368565412da7f5c78641dc357e => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:15.507056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:15.607027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612c54e-23c0-4244-b077-2951a5a94fa4 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:15.639670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:15.640269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:15.640550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:15.640821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:15.703893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:15.706115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:15.708087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:15.803170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 581/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:15.809149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:15.907268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e759d-23ec-45cb-a718-1c9377168d4b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:15.939120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:15.939757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:15.940020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:15.940294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] PATH_INFO: `/v3/projects/8d3ce521ae414da8896979c95813e115` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:15.941409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] RBAC: Authorizing `identity:get_project(project_id=8d3ce521ae414da8896979c95813e115)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:15.942874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba777d6-4fa4-429f-9801-8cb00c18a8de None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:15.944501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 579/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/projects/8d3ce521ae414da8896979c95813e115 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:15.951386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:16.046094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:16.049226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13a47-c956-44a4-b25d-7ce90342bf0d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:16.084145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:16.084747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:16.085030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:16.085302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:16.086024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=8d3ce521ae414da8896979c95813e115, user.id=12da3c368565412da7f5c78641dc357e) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:16.086333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:16.087888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fadb3-6c73-4e83-906b-ca1c69d0b2bc None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:16.093669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 582/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:26:15 2022] GET /identity/v3/role_assignments?user.id=12da3c368565412da7f5c78641dc357e&scope.project.id=8d3ce521ae414da8896979c95813e115&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:17.995541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43088039-7c63-4f87-a6c5-5f1b0acc8100 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:17.996091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43088039-7c63-4f87-a6c5-5f1b0acc8100 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:17.996091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43088039-7c63-4f87-a6c5-5f1b0acc8100 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:17.997414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 580/1162] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:18.002455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:18.002889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:18.003037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:18.021126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:18.022285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:18.086787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 583/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:18.359424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43088039-7c63-4f87-a6c5-5f1b0acc8100 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:18.387834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:18.460197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43088039-7c63-4f87-a6c5-5f1b0acc8100 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:18.492197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:18.492786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:18.493064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:18.493845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:18.557888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:18.559351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:18.587038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:18.660495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 581/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:18.666935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:18.755766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:18.762591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0f15f-684f-479a-bf42-c48d59de34cb None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:18.795226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:18.795831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:18.796107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:18.796374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:18.802705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:18.804134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b7e25721-e040-40a0-b216-70c5260a9429 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:18.816123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:18.817357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:18.818219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 584/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:26:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:18.824231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:18.922083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9948e-7dc7-4aca-a1e1-a96719f34282 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:18.947541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:18.956172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:18.956770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:18.957051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:18.957326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:18.957966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:18.958271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:18.959697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e60671-99c8-4a66-b58b-7273f904534f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:18.966848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 582/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 15:26:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:20.876198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e81ba-2aee-4c18-84f0-1f7275f9a9f7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.876782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e81ba-2aee-4c18-84f0-1f7275f9a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.876782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e81ba-2aee-4c18-84f0-1f7275f9a9f7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.878087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 585/1167] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:20.883500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e06ff-f91b-4794-8ee3-3b4007116efa None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.883994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e06ff-f91b-4794-8ee3-3b4007116efa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.884138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e06ff-f91b-4794-8ee3-3b4007116efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.902421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-694e06ff-f91b-4794-8ee3-3b4007116efa None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.903652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-694e06ff-f91b-4794-8ee3-3b4007116efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.968495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 583/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:21.235930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e81ba-2aee-4c18-84f0-1f7275f9a9f7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.335361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e81ba-2aee-4c18-84f0-1f7275f9a9f7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.369991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.370588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.370894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.371168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.371836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.373186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d152695-9c92-4776-8b0d-2afea8012d8b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.389451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 586/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:23.338025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142bb30-3989-43fa-8f78-8825441868b2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.338508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142bb30-3989-43fa-8f78-8825441868b2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.338611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142bb30-3989-43fa-8f78-8825441868b2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.339950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 584/1170] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:23.345114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.345535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.345666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.364032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.365210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.374912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.430149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 587/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:23.702127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142bb30-3989-43fa-8f78-8825441868b2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.800087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142bb30-3989-43fa-8f78-8825441868b2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.820110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.834612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.835269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.835554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.835830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] PATH_INFO: `/v3/domains/fe68b4a220724a3f91a78702f5422628` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.845915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe68b4a220724a3f91a78702f5422628)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.847384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2adc39-66f4-40ec-ac24-65fa938360de None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.848965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 585/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/domains/fe68b4a220724a3f91a78702f5422628 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:23.855077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.950584 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c4216-ee9e-41de-bed2-df380e9c4d99 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.983649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.984173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.984451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.984723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.985350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.986773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c29c8-894f-433c-8c29-b9f8f5afa99c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.001642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:24.014040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 588/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:25.960209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bde-9c5b-400f-a40b-d305190c62e1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.960740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bde-9c5b-400f-a40b-d305190c62e1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.960740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0211bde-9c5b-400f-a40b-d305190c62e1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.962627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 586/1174] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:25.967840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.968233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.968403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.987073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:25.988251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:26.054000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 589/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:26.334160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211bde-9c5b-400f-a40b-d305190c62e1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:26.434442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0211bde-9c5b-400f-a40b-d305190c62e1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:26.468878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.469486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.469760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.470030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] PATH_INFO: `/v3/domains/fe68b4a220724a3f91a78702f5422628` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.471156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe68b4a220724a3f91a78702f5422628)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.472576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb8edf-04cc-4656-9d5d-ac87a45e72dc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.474121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 587/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 15:26:26 2022] GET /identity/v3/domains/fe68b4a220724a3f91a78702f5422628 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:26.480223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:26.574827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad1855-b8c4-4e2d-b636-c0367cecdb27 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:26.598703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:26.607998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.608586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.608862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.609129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.609764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.611235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6a78b-49d1-4ada-b872-10a0be2a4f1d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.631845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 590/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:26:26 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:28.591566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe540f-ecba-43bf-846b-41148db93b69 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:28.592087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe540f-ecba-43bf-846b-41148db93b69 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:28.592087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe540f-ecba-43bf-846b-41148db93b69 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:28.593348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 588/1178] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:28.598587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:28.599059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:28.599154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:28.618324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:28.619537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:28.691416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 591/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:28.961634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe540f-ecba-43bf-846b-41148db93b69 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.011268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.062078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe540f-ecba-43bf-846b-41148db93b69 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.094995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.095616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.095899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.096173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.107372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:29.112044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:29.113054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:26:29.113945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 589/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:29.119271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.214992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc898-2505-4b2b-b618-8a2815844cf6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.228323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.248934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.249526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.249801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.250068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.250750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:29.251051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.252392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.260324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 592/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:29.266153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.364503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22ed2c-f557-4af4-9032-fd6b275615b3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.373088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.397175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.397924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.398884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.399215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] PATH_INFO: `/v3/users/747c12df04474b2a9d2ac7af10aa1ee5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.411795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] RBAC: Authorizing `identity:get_user(user_id=747c12df04474b2a9d2ac7af10aa1ee5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.413359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.415841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 590/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/users/747c12df04474b2a9d2ac7af10aa1ee5 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:29.421801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.517795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6debc-5dc9-4c18-be99-f86f58683adf None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.550366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.550978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.551257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.551529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.592901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.617814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.619385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.715270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 593/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:29.721178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.818390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d271766-ce1b-4d6d-a584-e0922929d214 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.852450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.853066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.853350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.853626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] PATH_INFO: `/v3/projects/772ec7ce26af408e844ca32c55fb56f0` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.854765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] RBAC: Authorizing `identity:get_project(project_id=772ec7ce26af408e844ca32c55fb56f0)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.856194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf7373-3a39-4a68-ba66-a2233686a896 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.857808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 591/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/projects/772ec7ce26af408e844ca32c55fb56f0 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:29.864035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.936921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.961927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5fe7-3c0c-4a7e-906c-4f07ec7d3602 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.995188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.995776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.996053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.996320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.997031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=772ec7ce26af408e844ca32c55fb56f0, user.id=747c12df04474b2a9d2ac7af10aa1ee5) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:29.997333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.998769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b606d07-c25b-483e-9110-1aa363a7f19d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:30.003940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 594/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/role_assignments?user.id=747c12df04474b2a9d2ac7af10aa1ee5&scope.project.id=772ec7ce26af408e844ca32c55fb56f0&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:32.024994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c420-b1e5-4356-b548-0c6210a07687 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.025553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c420-b1e5-4356-b548-0c6210a07687 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.025553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c420-b1e5-4356-b548-0c6210a07687 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.026964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 592/1186] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:32.032083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.032465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.032624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.051724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:32.052918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:32.118293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 595/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:32.394067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c420-b1e5-4356-b548-0c6210a07687 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.428882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:32.492971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c420-b1e5-4356-b548-0c6210a07687 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.525180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.525782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.526059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.526326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.535931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:32.541211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:32.542308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:26:32.543447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 593/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:32.548321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.612962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:32.645096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d47c-ba37-4403-bf50-7cc1c28e8e95 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.677807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.678400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.678714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.679014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.679674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:32.679981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.681336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.689133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 596/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:32.695843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.789575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:32.796670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3973a1-6ec4-43d7-9aff-7210226ffb6b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.829889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.830505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.830814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.831086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] PATH_INFO: `/v3/users/747c12df04474b2a9d2ac7af10aa1ee5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.832197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] RBAC: Authorizing `identity:get_user(user_id=747c12df04474b2a9d2ac7af10aa1ee5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.833672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.836301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 594/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/users/747c12df04474b2a9d2ac7af10aa1ee5 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:32.842167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.940217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc915b-98f7-4cf6-ac73-928bf1e516e2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.967717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:32.973804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.974421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.974731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.975011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] PATH_INFO: `/v3/projects/772ec7ce26af408e844ca32c55fb56f0` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.976440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] RBAC: Authorizing `identity:get_project(project_id=772ec7ce26af408e844ca32c55fb56f0)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.977863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14805a94-32be-4f5b-ad61-bb10e8e28890 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.979517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 597/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/projects/772ec7ce26af408e844ca32c55fb56f0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:32.985414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:33.083195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7757f-241e-4f85-831f-caf6a0d8deec None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:33.116080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:33.117339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:33.117622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:33.117901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] PATH_INFO: `/v3/projects/772ec7ce26af408e844ca32c55fb56f0/users/747c12df04474b2a9d2ac7af10aa1ee5/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:33.119809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] RBAC: Authorizing `identity:create_grant(project_id=772ec7ce26af408e844ca32c55fb56f0, user_id=747c12df04474b2a9d2ac7af10aa1ee5, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:33.121241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14601b-237e-45fb-ba9c-bc50ef5cada5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:33.135146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 595/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:26:32 2022] PUT /identity/v3/projects/772ec7ce26af408e844ca32c55fb56f0/users/747c12df04474b2a9d2ac7af10aa1ee5/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:35.029312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0acd3b-ba46-4275-829b-97d6484bfa6b None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.029868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0acd3b-ba46-4275-829b-97d6484bfa6b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.029868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0acd3b-ba46-4275-829b-97d6484bfa6b None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.031475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 598/1193] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:35.036533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.036946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.037082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.055884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:35.057061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:35.073192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:35.123045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 596/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:26:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:26:35.389708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0acd3b-ba46-4275-829b-97d6484bfa6b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:35.490122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0acd3b-ba46-4275-829b-97d6484bfa6b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:35.522170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:35.523331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.523613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.523889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.537864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:26:35.543906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:26:35.545058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:26:35.545910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 599/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:26:35.551313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:35.648307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee44aa5-1185-4c5d-9561-349baaa55a8a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:35.680434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:35.681027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.681309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.681583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.682244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:35.682547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:35.683946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:35.691708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 597/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:35.698350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:35.756625 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:35.798573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5894-ae3d-4e82-a66a-55448ea69b4e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:35.834197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:35.835532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.835861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.836129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] PATH_INFO: `/v3/users/747c12df04474b2a9d2ac7af10aa1ee5` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.837295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] RBAC: Authorizing `identity:get_user(user_id=747c12df04474b2a9d2ac7af10aa1ee5)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:35.838841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:35.840527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 600/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 15:26:35 2022] GET /identity/v3/users/747c12df04474b2a9d2ac7af10aa1ee5 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:35.846515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:35.863615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:35.946426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326b3a-b6cb-4521-bba0-c0719eaf04f0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:35.978503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:35.979131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.979414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.979691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.044274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:36.045714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:36.059984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:36.142515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 598/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:26:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:26:36.148986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:36.250282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7a6a2-9eab-4f0b-b866-b9fc71c4ad3a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:36.275588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:36.284358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:36.284984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.285265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.285538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] PATH_INFO: `/v3/projects/772ec7ce26af408e844ca32c55fb56f0` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.286663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] RBAC: Authorizing `identity:get_project(project_id=772ec7ce26af408e844ca32c55fb56f0)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:36.288665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5183ec-2712-42dc-b5d6-e6ea7ac6badd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:36.290310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 601/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 15:26:36 2022] GET /identity/v3/projects/772ec7ce26af408e844ca32c55fb56f0 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:26:36.296607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:36.394019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb985-255c-4ff1-9271-8ea741da4fcc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:36.407496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:36.427382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:36.427975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.428246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.428569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.430094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=772ec7ce26af408e844ca32c55fb56f0, user.id=747c12df04474b2a9d2ac7af10aa1ee5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:36.430401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:36.431905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9347723-5fe7-4abc-9bdc-a6d027f23640 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:36.437435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 599/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:26:36 2022] GET /identity/v3/role_assignments?user.id=747c12df04474b2a9d2ac7af10aa1ee5&scope.project.id=772ec7ce26af408e844ca32c55fb56f0&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:46.280293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:46.434938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=11 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:31.793488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ebfc-335d-493d-9fc2-9c7baa0ef46e None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:31.794148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ebfc-335d-493d-9fc2-9c7baa0ef46e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:31.794256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3ebfc-335d-493d-9fc2-9c7baa0ef46e None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:31.795742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 602/1201] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:27:31.801521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:31.802055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:31.802055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:31.824766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:31.825997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:31.907935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 600/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:27:32.242468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3ebfc-335d-493d-9fc2-9c7baa0ef46e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:32.411401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3ebfc-335d-493d-9fc2-9c7baa0ef46e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:32.468866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:32.469479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:32.469758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:32.470186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:32.484217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:32.485797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:32.505027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:32.506707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:32.508125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:32.508125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:32.508125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:32.508125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 603/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:32.513337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:32.693886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c330c4f-44f9-4b67-9256-79fdcf40661d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:32.748661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:32.757831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:32.758470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:32.758930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:32.759209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:32.760116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:32.760427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:32.763126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a82883-4849-4db3-9f4d-7df420b3ec2a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:32.787874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 601/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:32.789229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:32.865559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:32.941791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9969e8-7b9d-4107-ac9e-27f97deb440a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:32.992134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:32.993206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:32.994023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:32.994803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:32.996359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:32.999216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1912a80-ef09-42dd-a5e3-31f753478586 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:33.040880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 604/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 15:27:32 2022] POST /identity/v3/users => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:35.249366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737cff-c3ec-496c-91ca-6a363ee34ee9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.249967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737cff-c3ec-496c-91ca-6a363ee34ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.249967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01737cff-c3ec-496c-91ca-6a363ee34ee9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.251449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 602/1206] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:27:35.257516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.257986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.258081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.278474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:35.279847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.370889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 605/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:27:35.661550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737cff-c3ec-496c-91ca-6a363ee34ee9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.781102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01737cff-c3ec-496c-91ca-6a363ee34ee9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:35.821123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:35.822075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.822562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.823063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.836460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:35.842511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:35.843755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:35.844640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 603/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:27:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:35.853048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.902925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:35.980747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5c3c8-2a5d-4113-86af-5c0a69ebad98 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.025100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.025765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.026042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.026310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.027068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:36.027370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.030848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.045425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 606/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:27:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:36.046702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.119133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.190983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78ae20-2c52-4441-972f-5c8b289643f4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.235103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.235866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.236243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.236594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.249209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.253206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:36.271924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:36.273155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:36.274046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:36.274046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:36.274046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:36.275400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 604/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:36.281423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.413975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.418508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67ff2-2ff5-4652-93af-7c5cbccbf12d None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.465184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.465929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.466315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.466682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.467448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:36.467766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.469430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.482898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 607/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:36.490380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.610226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a80fe-8222-4f78-b6de-f0c6d466011d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.612284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.658311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.659103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.660181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.660484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.669950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.671648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:27:36.684531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:36.685628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:36.693633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 605/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:36.694268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.833780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea9ec-d9a1-4655-8e33-a9b0d705f3af None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.883372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.884200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.884497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.884880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.885883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:36.886307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.887950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.914685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 608/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:36.916962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.038741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401151-1ad5-43d5-8d0b-469198b0c1cc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.077065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.077687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.077968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.078244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.112608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.155771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.157235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.288027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 606/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:27:37.290323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.316908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.413135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f468cc-d500-4c93-ac4e-1bc4d7ffbca5 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.457304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.458328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.458619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.458920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.469802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.471294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:37.488114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:37.489198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:37.490161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:37.490161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:37.490161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:37.494723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 609/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:37.495160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.560199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.602166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac24d6-119a-419a-b331-bba264559b68 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.638882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.639516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.639801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.640056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.640718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:37.641026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.642383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.655174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 607/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:37.663361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.746410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.785042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff258d-94b0-44cf-aab9-9586e652b1f0 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.822237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.822941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.823263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.823540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.831708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.833302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:27:37.847516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:37.849066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:37.850000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:37.854924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.856049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 610/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:37.973332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22decb8-92a7-4293-bed5-8e581b7d17dd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.005075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:38.017865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.018995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.021127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.021127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.021127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:38.021127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.022068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821d91e-e2eb-4473-a6c9-3650a71112ad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.035456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 608/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:38.041612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.162878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff0005-dc6d-4bd4-95c7-2966905f9cea None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.196040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:38.203864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.204512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.204803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.205207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.206415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=5edde502e37144e1b36c7fcfadb6acdd) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:38.207033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.208532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e90a1-98a8-4a9a-ab2e-c8560f43c532 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.214665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 611/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:27:38 2022] GET /identity/v3/role_assignments?user.id=5edde502e37144e1b36c7fcfadb6acdd&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:40.460230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4100cb-7078-473b-a7fd-9ced553c116e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.460771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4100cb-7078-473b-a7fd-9ced553c116e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.460771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4100cb-7078-473b-a7fd-9ced553c116e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.462694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 609/1220] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:40.467538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.467948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.468104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.486504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:40.488289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.559565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 612/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:27:40.832196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4100cb-7078-473b-a7fd-9ced553c116e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.946665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4100cb-7078-473b-a7fd-9ced553c116e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.981345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.982000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.982279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.982583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.994198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:40.999342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:41.001054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:41.002380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 610/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:41.006930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.116174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118c0c-cacf-446c-86cd-c872649313a5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.140536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.152952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.153614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.153894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.154170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.154969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] RBAC: Adding query filter params (name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:41.155279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.156651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.166063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 613/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:41.172096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.219737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.286517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb66f-2fae-4f54-b49d-044aa5fee72e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.337193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.337896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.338181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.338453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.349025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.350480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:41.364965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:41.365979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:41.366967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:41.366967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:41.366967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:41.366967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 611/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:41.372069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.496206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bda58-210f-4e23-a0a9-ec909f118742 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.536483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.537209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.537526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.537815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.538485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:41.538842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.540318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.555024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 614/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:41.561066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.669551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.677898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeb6ab-7c57-4355-bec8-8659887ba8a3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.714713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.715349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.715633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.715910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.724118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.725635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:27:41.738771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:41.739912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:41.740944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 612/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:41.745730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.803580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.867672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcacc5f-7087-4a87-a15b-197391fe7187 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.909316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.910051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.910340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.910614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.911412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:41.911723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.913248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.936092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 615/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:41.941923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.051854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5e98e-fda4-4a87-8642-0b5c25ca8007 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.089024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.089658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.089943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.090221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.101800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.103321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:42.120638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:42.121784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:42.122931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:42.122931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:42.122931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:42.122931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 613/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:42.128436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.234870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e52552-bb5d-41ac-9188-d9f7c4cfa9cf None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.238501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.275153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.275824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.276053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.276357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.277093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:42.277411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.278946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.291847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 616/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:42.297747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.310180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.419118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a162a4-9ee6-43ac-9f60-22ba8a45ba0a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.459898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.460562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.460820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.461095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.468176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.469617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:27:42.481674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:42.482681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:42.483592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 614/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:42.489371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.599155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00e8d-49bc-494c-bb61-9d7e863acabd None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.643897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.645038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.645379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.645780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.646898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:42.647254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.648979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde672c-c240-4b1b-9b7c-e6ee930b34e6 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.662716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 617/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:42.669157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.743553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.782117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8cbc5-44f9-4d45-8053-3643b48d2e45 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.820994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.821693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.822046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.822325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/5edde502e37144e1b36c7fcfadb6acdd/roles/9266a3b12c574b73b8b30e2876798086` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.839581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=5edde502e37144e1b36c7fcfadb6acdd, role_id=9266a3b12c574b73b8b30e2876798086)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.841281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760473b-e065-4e97-abb9-a42cad031e70 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.856133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 615/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/5edde502e37144e1b36c7fcfadb6acdd/roles/9266a3b12c574b73b8b30e2876798086 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:44.854270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f643-2962-46fe-be06-9c2baf78fd4b None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.855127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f643-2962-46fe-be06-9c2baf78fd4b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.855311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394f643-2962-46fe-be06-9c2baf78fd4b None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.857215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 618/1233] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:44.862771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.863216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.863340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.883072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:44.884414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:44.958760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 616/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:27:45.271784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394f643-2962-46fe-be06-9c2baf78fd4b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.299459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.392069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394f643-2962-46fe-be06-9c2baf78fd4b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.430847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.431482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.431767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.432036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.442563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:45.447376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:45.448477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:45.449707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 619/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:45.454635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.503337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.567848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e2213-1dfc-4242-b317-3005170e9339 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.614053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.614716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.615078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.615359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.616040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:45.616371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.617878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:45.627074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 617/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:45.636187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.729674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.746282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d3c0-3b75-4a11-81b6-d0fa19d77a9a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.782718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.783359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.784263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.784485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.795452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.796900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:45.812836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:45.813872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:45.815219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:45.815219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:45.815219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:45.815219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 620/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:45.820006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.944565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd743c2f-fb0b-49c2-95ff-8f2bb0d5cf18 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.954321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.983284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.983936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.984287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.984569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.985245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:45.985558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.987138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.000976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 618/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:46.007876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.116792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6bb7f-7ed1-4eb0-aa86-883e0c003a38 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.151390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.152018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.152304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.152586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.155326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.161212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.162690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:27:46.173869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:46.175110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:46.176113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 621/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:46.181146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.294049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cac32d-8854-4283-8c3a-14b72125422d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.332964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.333549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.333824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.334096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.334998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:46.335282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.336834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.355725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 619/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:46.362489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.478009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c397957-47be-490f-8eea-46c675da5b2f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.515526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.516143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.516466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.516705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.595490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.603185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.604721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.710050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 622/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:27:46.716120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.744555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.828771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b445fb1-2062-47fa-ad97-cf8c15e95ebb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.872304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.873241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.873681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.874140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.885415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.887003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:46.903633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:46.910290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:46.911542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:46.911542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:46.911542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:46.911542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 620/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:46.911849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.015511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:47.025219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dd2b5-8e59-4449-80d8-b15be2f2e5ae None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.062175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.062904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.063257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.063607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.064332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:47.064632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.066046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.077361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 623/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:47.083629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.171929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:47.196480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f4905-5498-4639-b12b-f981d16cdad8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.234912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.235547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.235829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.236247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.243276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.244725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:27:47.257786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:47.258921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:47.264782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 621/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:47.265509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.378082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dd387-81c8-42fd-b60e-45f476fb36f2 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.418954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.419660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.420014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.420359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.421123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:47.421431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.422984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d8cec-199f-4a65-9a40-f571d2ef91cb None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.443731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 624/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:47.450160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.575761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6ab3f-a24f-4f64-bfd2-f360c0ac31b2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.611333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.612524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.612797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.613092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.613809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] RBAC: Adding query filter params (role.id=9266a3b12c574b73b8b30e2876798086, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=5edde502e37144e1b36c7fcfadb6acdd) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:47.614130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.615595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a3c40-0586-4583-98ba-8ed5ffddd849 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.634671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 622/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:27:47 2022] GET /identity/v3/role_assignments?user.id=5edde502e37144e1b36c7fcfadb6acdd&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=9266a3b12c574b73b8b30e2876798086 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:49.784204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f22d-c5c3-404c-9701-9d9610f47ca2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.785027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f22d-c5c3-404c-9701-9d9610f47ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.785027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f22d-c5c3-404c-9701-9d9610f47ca2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.787014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 625/1247] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:49.795010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.795569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.795569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.802906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.817600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.818951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.910540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 623/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:27:50.251593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5f22d-c5c3-404c-9701-9d9610f47ca2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.277005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.369788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5f22d-c5c3-404c-9701-9d9610f47ca2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.412162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.413108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.413422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.413693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.424367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:50.429731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:50.431808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:50.433010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:50.434162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 626/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:50.440985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.552815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eae7c-b81c-48c2-a763-461b54ca8316 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.583974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.589420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.590031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.590311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.590575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.591345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:50.591624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.593163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.602944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 624/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:50.608907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.697043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.722589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd43df-51b9-4f73-b2d3-0ce6baf788ab None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.761343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.761981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.762272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.762532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.775066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.776537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:50.792597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:50.794012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:50.794998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:50.794998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:50.794998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:50.794998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 627/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:27:50.800260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.922289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe0bcb-185c-485d-88b9-8d9802950a1e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.963131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.963753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.964172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.964560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.965216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:50.965665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.967580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.983130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 625/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:50.989547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.119374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b60b-64e2-42c9-b171-1fa6ff3d0682 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.151236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.163381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.164016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.164301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.164577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.172453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.173826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:27:51.185492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:51.186508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:51.187523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 628/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:51.192614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.258765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.344748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8107b22-3d2b-4a44-93dd-d626964858cf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.393330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.394065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.394346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.394623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.395385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:51.395695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.397183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.416332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 626/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:27:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:51.422778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.552112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281d52b-04c2-48e8-879e-c558f6c1860a None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.589601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.590211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.590496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.590807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.660532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.673797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.675697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.792429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 629/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:27:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 15:27:51.798201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.935158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752acec9-fe3a-4da8-82b1-01df2576bf6b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.950135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.972734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.973364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.973896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.974170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.983995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.985429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:52.001938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:52.003222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:52.006080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:52.006080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:52.006080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:52.006080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 627/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:52.009716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.140408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:52.153795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5ebae-e653-4657-824d-f56e32eeab6f None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.191442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.192004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.192283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.192558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.193376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:52.193689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.195093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.207630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 630/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:52.213461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.322564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad9af0-6d01-4ba9-a24e-702acee06b34 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.361229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.361667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.362486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.362803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.369704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.371242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:27:52.383472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:52.384588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:52.390835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 628/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:52.391429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.514817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8625-ed63-400c-83bc-3b3e7303d776 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.565030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.565675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.565953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.566227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.567057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:52.567310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.569056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ffc80-0009-4cfb-86b6-e9331fc85ca8 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.585864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 631/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:52.595401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.634295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:52.736812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34866ec7-25c6-478a-b346-b63fb630e1b1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.776615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.777157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.777437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.777712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.778467 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=5edde502e37144e1b36c7fcfadb6acdd) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:52.778798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.780277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c8e7b-8c56-40c3-abd2-6693966dba8d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.787010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 629/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/role_assignments?user.id=5edde502e37144e1b36c7fcfadb6acdd&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:54.794511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a9d25-3c12-46fc-a71c-00cbe4308617 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.795070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a9d25-3c12-46fc-a71c-00cbe4308617 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.795163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a9d25-3c12-46fc-a71c-00cbe4308617 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.796451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 632/1261] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:54.801994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.802394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.802545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.821778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:54.822985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.898333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 630/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:27:55.167847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a9d25-3c12-46fc-a71c-00cbe4308617 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.179496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.270193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a9d25-3c12-46fc-a71c-00cbe4308617 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.313982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.314616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.315027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.315903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.326761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:55.331964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:55.333092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:55.334149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 633/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:27:55.339056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.357454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.437982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e01ee3-19e7-4efd-82d9-0c605002da23 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.471958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.472721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.473003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.473274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.474190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:55.474488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.475890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.484018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 631/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:55.490542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.558465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.590650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75389c-612d-4217-9807-ee5d033cddf0 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.625390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.626003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.626282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.626554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.636474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.637899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:55.652402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:55.653562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:55.654480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:55.654480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:55.654480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:55.654480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 634/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:55.661489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.749656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.768945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead9528-f1e5-4c09-b52f-065569e93d1a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.802369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.803062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.803270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.803548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.804185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:55.804487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.805908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.818108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 632/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:55.823847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.937553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7b42f-e68c-4a01-8f05-b05fc7033c1c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.958414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.973993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.974624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.974934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.975206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.982662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.984117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:27:55.995064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:55.996131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:55.997537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 635/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:56.002296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.128102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d627198-5959-47c7-beda-f1ce0f5bdf08 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.162495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.163125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.163407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.163683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.164410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:56.164716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.166148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.174571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.183136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 633/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:56.188799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.287421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a10e36-3ed7-481a-be20-e8bc2dc9807b None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.321513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.322140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.322416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.322725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.334132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.335611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:56.350046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:56.351233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:56.352199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:56.352199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:56.352199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:56.352199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 636/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:56.357223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.454064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfdd74-ca72-4e5f-b9e4-98ff15e715bf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.487911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.488517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.488799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.489074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.489722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:56.490033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.491431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.503262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 634/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:56.508926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.540722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.607642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ad1ea-a653-4b98-94fb-8afd90235e4e None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.640458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.641070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.641349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.641621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.648007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.649422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:27:56.660477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:56.661498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:27:56.662792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 637/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:56.667472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.726337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.767043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4f2c-71dd-40b4-9c69-892612e6da5e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.798475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.799035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.799828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.800098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.800780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:56.801080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.802488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af751f-36b6-4dfe-8f09-599c272f25ea None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.813845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 635/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:56.820478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.920118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.929353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674df6c-e6e2-4ae9-b510-494f660e9321 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.962140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.962826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.963107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.963377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] PATH_INFO: `/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/5edde502e37144e1b36c7fcfadb6acdd/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.965247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] RBAC: Authorizing `identity:create_grant(project_id=e025a9fe2c2249c99087d3d22603d623, user_id=5edde502e37144e1b36c7fcfadb6acdd, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.966705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec6e9d-4c24-4ae9-9493-d5e3f4fe915b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.979334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 638/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 15:27:56 2022] PUT /identity/v3/projects/e025a9fe2c2249c99087d3d22603d623/users/5edde502e37144e1b36c7fcfadb6acdd/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:58.876775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc342fd-c35c-47b4-8845-ab866fde9640 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.877340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc342fd-c35c-47b4-8845-ab866fde9640 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.877432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc342fd-c35c-47b4-8845-ab866fde9640 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.878795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 636/1274] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:58.883937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.884339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.884494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.902798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:58.903975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:58.967972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 639/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:27:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:27:59.233640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc342fd-c35c-47b4-8845-ab866fde9640 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.333406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc342fd-c35c-47b4-8845-ab866fde9640 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.350877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:59.367904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.368521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.368796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.369065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.378960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:27:59.383593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:59.384705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:59.385582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 637/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 15:27:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:59.390595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.468265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:59.487714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eaf02-0020-43fd-baf8-04b1aafd4bdf None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.520352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.520951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.521231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.521503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.522158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:59.522461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:59.523851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:59.531598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 640/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 15:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:59.538729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.639957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f808807-e00e-4b7f-b168-d989bfea490b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.673974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.674616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.674929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.675206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.690193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:59.692779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:27:59.711714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:27:59.712878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:27:59.714405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:27:59.714405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:27:59.714405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:27:59.714405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 638/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:27:59.718786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.815679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c29-c8d5-496c-b677-1ea4451fc338 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.852163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.852852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.853102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.853406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.854123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:59.854455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:59.855980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:59.869508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 641/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:59.876166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.912534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:59.982111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010ad79-b871-45b1-8fcd-8068240d8801 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.015264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951a03-6f0a-4096-aded-92680048efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.015889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951a03-6f0a-4096-aded-92680048efde None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.016171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951a03-6f0a-4096-aded-92680048efde None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.016447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54951a03-6f0a-4096-aded-92680048efde None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.024328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54951a03-6f0a-4096-aded-92680048efde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.025810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54951a03-6f0a-4096-aded-92680048efde None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-54951a03-6f0a-4096-aded-92680048efde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:28:00.036847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:00.037935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:00.039109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 639/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:27:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:00.044008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.047823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.141946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ee67f-9fd6-49af-b8ab-56ec80f01c0c None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.175029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.175624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.175905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.176177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.176903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:00.177212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.178629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.195591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 642/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:00.201274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951a03-6f0a-4096-aded-92680048efde None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.290524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.300637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54951a03-6f0a-4096-aded-92680048efde None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.335489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.336097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.336380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.336656 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.402600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.404014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.489671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.500046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 640/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:28:00.505898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.566143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.605464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f3d0-ba1b-4efe-bd7d-9431a1c3f843 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.638358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.638992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.639268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.639535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.649750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.651192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:28:00.665688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:00.666852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:28:00.667800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:28:00.667800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:28:00.667800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:28:00.667800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 643/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:00.672427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.768742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62babda-2768-44c6-a5bd-925a8a873251 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.801214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.801810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.802093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.802368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.803032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:00.803339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.804690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.816206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 641/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:00.821791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.924171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40adb20-e929-4f1e-805b-0cabd4073098 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.937272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.958930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.959657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.959938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.960214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.966694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.968126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:28:00.979467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:00.980507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: ERROR keystone.server.flask.application Aug 24 15:28:00.981468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 644/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:00.986391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.036223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:01.085153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b25772-6d1a-475b-82ef-b0579e7f2788 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.117529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.118118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.118401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.118697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.119384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:01.119689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.121100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a399712-14f8-4efd-8914-25a97570d6c2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.133068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 642/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:01.139101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.235031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04fb44-271e-45ef-8697-bdb721a13398 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.267697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.268371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.268647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.268917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.269631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] RBAC: Adding query filter params (role.id=826e2155943c48ab9c01f294b1734d83, scope.project.id=e025a9fe2c2249c99087d3d22603d623, user.id=5edde502e37144e1b36c7fcfadb6acdd) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:01.269936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.271371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12dfe7-a57e-419c-8007-5aa324153c2b None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.276887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 645/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 15:28:01 2022] GET /identity/v3/role_assignments?user.id=5edde502e37144e1b36c7fcfadb6acdd&scope.project.id=e025a9fe2c2249c99087d3d22603d623&role.id=826e2155943c48ab9c01f294b1734d83 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:03.196619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdca6-f940-443a-9859-139f6fce6ccf None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.197177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdca6-f940-443a-9859-139f6fce6ccf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.197177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdca6-f940-443a-9859-139f6fce6ccf None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.198505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 643/1288] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:28:03.203607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.203997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.204159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.222347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:03.223559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:03.251978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:03.289123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 646/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:28:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:28:03.557067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdca6-f940-443a-9859-139f6fce6ccf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:03.656862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdca6-f940-443a-9859-139f6fce6ccf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:03.667170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:03.691221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:03.691846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.692128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.692403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.692985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:03.694352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:03.702926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:03.703943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:03.705054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 644/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:28:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:03.710077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:03.809671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3641e3-7233-43fc-b1a1-2643b6a0ffe7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:03.843214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:03.843799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.844083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.844374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.844999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:03.845301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:03.847230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6012-d4e4-4a37-b2b9-321f6d14a697 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:03.852690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 647/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 15:28:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:03.858369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:03.970286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12e3cc-6a2f-45e5-8cbb-7e31c721e5dc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:04.003806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:04.004455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:04.004736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:04.005011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:04.005633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:04.005934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:04.007325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a9f0-a1b6-49fc-aacd-43395a8df448 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:04.011632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 645/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 15:28:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:05.898629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244df8-9c29-4d6e-8063-d8856fe21dcf None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.899181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244df8-9c29-4d6e-8063-d8856fe21dcf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.899273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244df8-9c29-4d6e-8063-d8856fe21dcf None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.900547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 648/1293] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:28:05.905778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b04c3-4d0b-40da-8a07-0f7c72acc52c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.906178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b04c3-4d0b-40da-8a07-0f7c72acc52c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.906329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b04c3-4d0b-40da-8a07-0f7c72acc52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.924820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-050b04c3-4d0b-40da-8a07-0f7c72acc52c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:05.926092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-050b04c3-4d0b-40da-8a07-0f7c72acc52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:05.954917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:05.993699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 646/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:28:06.267103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40244df8-9c29-4d6e-8063-d8856fe21dcf None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:06.298074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:06.369060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40244df8-9c29-4d6e-8063-d8856fe21dcf None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:06.402361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:06.402980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:06.403259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:06.403531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:06.404177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:06.405525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cd71-1f6c-4c3d-90ec-036ef4ba5912 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:06.419229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 649/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 15:28:06 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:08.362242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa0db-61f4-4ce4-b3e3-36fd98793dcf None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:08.362758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa0db-61f4-4ce4-b3e3-36fd98793dcf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:08.362876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa0db-61f4-4ce4-b3e3-36fd98793dcf None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:08.364171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 647/1296] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:28:08.369405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:08.369776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:08.369950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:08.388171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:08.389336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:08.460664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 650/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:28:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:28:08.731698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aa0db-61f4-4ce4-b3e3-36fd98793dcf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:08.830071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aa0db-61f4-4ce4-b3e3-36fd98793dcf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:08.868922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:08.869491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:08.869813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:08.870037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:08.870619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:08.872007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:08.881006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:08.882083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:08.882976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 648/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:28:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:08.888467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:08.909634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:08.989441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fff4f-3811-4d68-9597-a42ff192ef06 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:09.022572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:09.023190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:09.023470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:09.023745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:09.024371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:09.024676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:09.026028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:09.031000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 651/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 15:28:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:09.037445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:09.068709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:09.136802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802d38-57f3-48d6-abd7-e1e0cd8b004f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:09.169557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:09.170156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:09.170437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:09.170741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:09.171442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] RBAC: Adding query filter params (interface=public, service_id=95a411024e944e67b1a55a07cbe658a4, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:09.171755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:09.173119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ecdb4-cf85-4f8b-929c-38785e7c852d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:09.178024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 649/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 15:28:09 2022] GET /identity/v3/endpoints?service_id=95a411024e944e67b1a55a07cbe658a4&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:28:09.183695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:09.262646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:09.280595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54d40-b661-4241-a499-5600213d3024 None admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:09.313270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:09.313863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:09.314140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:09.314411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:09.315627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:09.317001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe8fe6-89b3-4cc5-ac8f-f519e656bdaa None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:09.324425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 652/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 15:28:09 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:11.235635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7cc1-6577-48e3-b657-cc050ec1117a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.236210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7cc1-6577-48e3-b657-cc050ec1117a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.236210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7cc1-6577-48e3-b657-cc050ec1117a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.237825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 650/1302] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:28:11.243143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.243559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.243688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.262047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:11.263262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:11.329974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 653/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:28:11.601114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7cc1-6577-48e3-b657-cc050ec1117a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.701279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:11.708744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7cc1-6577-48e3-b657-cc050ec1117a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.742298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.742944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.743225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.743498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.744075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.745428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:28:11.754529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:28:11.755740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Aug 24 15:28:11.756792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 651/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:28:11.761662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.839793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:11.867802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751777c1-e93e-400e-a488-d634c3e38ace None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.901303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.901901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.902181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.902457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] PATH_INFO: `/v3/services` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.903117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:11.903422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.904776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6adcb2-6867-486f-9b1e-08f5727d28c9 None admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.910004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 654/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:11.915822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:12.013819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec6494-ad32-4305-bdb2-9a7ec3bb95db None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:12.047361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:12.047960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:12.048241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:12.048513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:12.049173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:12.051234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ece70-471c-46f2-ac9b-c6104c986854 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:12.070102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 652/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:21.907215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=30 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:22.065798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:16.420996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd99b-7887-4b27-8643-2c942373f268 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:16.422045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd99b-7887-4b27-8643-2c942373f268 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:16.422537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dd99b-7887-4b27-8643-2c942373f268 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:16.424588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 655/1307] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:16.429952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b194-e8c4-40c2-b07b-666a2cbed2a5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:16.430417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b194-e8c4-40c2-b07b-666a2cbed2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:16.430538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b194-e8c4-40c2-b07b-666a2cbed2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:16.523304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e487b194-e8c4-40c2-b07b-666a2cbed2a5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:16.524904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e487b194-e8c4-40c2-b07b-666a2cbed2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:16.659659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 653/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:16.965596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:16.965890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:16.966177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:16.967533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 656/1309] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 15:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:16.974198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0da6dc-1709-4c1c-8f09-f95bd6eea871 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:16.974493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0da6dc-1709-4c1c-8f09-f95bd6eea871 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:16.974923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0da6dc-1709-4c1c-8f09-f95bd6eea871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:17.076305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7b0da6dc-1709-4c1c-8f09-f95bd6eea871 None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:17.077634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7b0da6dc-1709-4c1c-8f09-f95bd6eea871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:17.100447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:17.161409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 654/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:17.170026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:17.171949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:17.320209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918391b3-86b5-4dfb-9df0-593f60f180a7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:17.414695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:17.415233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:17.415510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:17.415782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:17.417397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:17.475654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:17.534139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:17.535630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727a30-f28a-4051-b1c7-5bb3b261baed service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:17.555700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 657/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:29:19.790936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:19.791412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:19.791517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:19.793401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 655/1312] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:19.798240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8ba5-d067-4ecc-9048-94818215cc32 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:19.798670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8ba5-d067-4ecc-9048-94818215cc32 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:19.798836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8ba5-d067-4ecc-9048-94818215cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:19.881801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-843f8ba5-d067-4ecc-9048-94818215cc32 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:19.883043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-843f8ba5-d067-4ecc-9048-94818215cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:19.981252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 658/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:20.268886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:20.270441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:20.378035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdee3-8f21-49d2-b191-cce589822e98 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:20.453088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:20.455733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:20.456559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:20.456893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:20.457331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:20.459751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:20.567313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:20.568752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10819f2a-8ad4-4560-b10f-aa165d607e22 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:20.587081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 656/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:22.842235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:22.842898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:22.842898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:22.844676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 659/1315] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:22.851405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353716bd-3e48-47e4-9789-0213c428e892 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:22.851848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353716bd-3e48-47e4-9789-0213c428e892 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:22.851946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353716bd-3e48-47e4-9789-0213c428e892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:22.918092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-353716bd-3e48-47e4-9789-0213c428e892 None None] MFA Rules not processed for user `1a7f29f505094c969219499c619c7462`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:22.919352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-353716bd-3e48-47e4-9789-0213c428e892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:22.972096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:23.016399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 657/1316] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:23.302590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:23.304161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:23.314051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:23.410220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e9ef-eb6c-41dd-8011-ff53c4c6a123 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:23.483426 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:23.484030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:23.484409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:23.484688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:23.486298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:23.548667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:23.596831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:23.598262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f06098-b704-4488-8211-314a813c7271 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:23.616960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 660/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:26.062193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:26.062762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:26.062762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:26.064113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 658/1318] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:26.069781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fb8-7ed6-447f-b9ee-2242ecef513f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:26.070196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fb8-7ed6-447f-b9ee-2242ecef513f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:26.070360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6fb8-7ed6-447f-b9ee-2242ecef513f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:26.135843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-32ef6fb8-7ed6-447f-b9ee-2242ecef513f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:26.137049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-32ef6fb8-7ed6-447f-b9ee-2242ecef513f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:26.231830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 661/1319] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:26.530369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:26.531992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:26.639286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2847ec-3e36-47a8-8d1f-8771aabc08e0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:26.715429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:26.716036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:26.716315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:26.716591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:26.718203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:26.726759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:26.829489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:26.830967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646a35-4900-4dee-8f8f-c8cb36171b67 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:26.850768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 659/1320] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:29.386095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:29.386669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:29.386669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:29.388027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 662/1321] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:29.393365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51fdb-aa7e-477a-ab47-aa5c7a4abc74 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:29.393856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51fdb-aa7e-477a-ab47-aa5c7a4abc74 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:29.393956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d51fdb-aa7e-477a-ab47-aa5c7a4abc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:29.459363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-01d51fdb-aa7e-477a-ab47-aa5c7a4abc74 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:29.460581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-01d51fdb-aa7e-477a-ab47-aa5c7a4abc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:29.517461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:29.558520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 660/1322] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:29.851241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:29.852785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:29.873520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:29.960810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37cbed-9766-48f2-82c9-ba269aa4f210 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:30.045744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:30.046351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:30.046630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:30.046932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:30.048508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:30.120484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:30.159362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:30.160810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b517-0847-4fa6-a322-4b97c8f7efb5 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:30.179732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 663/1323] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:32.186343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd6bf-e14a-4a1e-b774-e8e327142771 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:32.186898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd6bf-e14a-4a1e-b774-e8e327142771 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:32.186898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd6bf-e14a-4a1e-b774-e8e327142771 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:32.188200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 661/1324] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:32.194349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3482d-79c0-4524-ade4-e8047699ed4f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:32.194876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3482d-79c0-4524-ade4-e8047699ed4f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:32.195026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3482d-79c0-4524-ade4-e8047699ed4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:32.261096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ebb3482d-79c0-4524-ade4-e8047699ed4f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:32.262302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ebb3482d-79c0-4524-ade4-e8047699ed4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:32.361330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 664/1325] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:32.661093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:32.661379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:32.661710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:32.663514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 662/1326] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:32.669565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361307-9b29-40ba-8a4f-bb4d4aca822f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:32.669950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361307-9b29-40ba-8a4f-bb4d4aca822f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:32.670302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361307-9b29-40ba-8a4f-bb4d4aca822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:32.702702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:32.748555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-14361307-9b29-40ba-8a4f-bb4d4aca822f None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:32.758141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-14361307-9b29-40ba-8a4f-bb4d4aca822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:32.852850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 665/1327] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:32.861929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:32.863731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:32.967907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9c243-fa59-45d9-8602-8945cd88d5f7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:33.042485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:33.043031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:33.043312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:33.043586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:33.045188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:33.062355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:33.153760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:33.155248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3648b93-7eb1-4b27-ac41-c00a9c96b566 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:33.174086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 663/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:35.404980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:35.405647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:35.405647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:35.407773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 666/1329] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:29:35.415455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb23bb-8484-40b4-9592-bdce56b75baf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:35.415926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb23bb-8484-40b4-9592-bdce56b75baf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:35.416006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb23bb-8484-40b4-9592-bdce56b75baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:35.494602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b5eb23bb-8484-40b4-9592-bdce56b75baf None None] MFA Rules not processed for user `1a7f29f505094c969219499c619c7462`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:35.495890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b5eb23bb-8484-40b4-9592-bdce56b75baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:35.565331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:35.600816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 664/1330] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:35.964917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:35.967308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:36.118323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6fdf-1565-4426-92ac-309e1d63f29c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:36.147039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:36.216251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:36.216565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:36.216800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:36.217077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:36.218801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:36.357847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:36.359366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05855844-a6b8-4a2c-8efc-9b3e982fc26b service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:36.379474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 667/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:38.712847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:38.713360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:38.713360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:38.714688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 665/1332] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:38.720093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd702c7-feb1-44c4-abe6-206e125169a3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:38.720525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd702c7-feb1-44c4-abe6-206e125169a3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:38.720656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd702c7-feb1-44c4-abe6-206e125169a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:38.784540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:38.787852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9bd702c7-feb1-44c4-abe6-206e125169a3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:38.789720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9bd702c7-feb1-44c4-abe6-206e125169a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:38.886432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 668/1333] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:39.167438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.168983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.273964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89a9fe-d520-43fa-a0d4-db58bb61f8d5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.346810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.347418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.347698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.347969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.349568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.378133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:39.514097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.515571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d0e9-e039-4473-a5a1-4c1bb34c7094 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.533441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 666/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:41.489800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:41.490489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:41.490489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:41.491779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 669/1335] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:41.496922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292254-a480-49e6-8bf5-41284541749e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:41.497342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292254-a480-49e6-8bf5-41284541749e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:41.497469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292254-a480-49e6-8bf5-41284541749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:41.563948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e9292254-a480-49e6-8bf5-41284541749e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:41.565164 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9292254-a480-49e6-8bf5-41284541749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:41.640267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:41.662392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 667/1336] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:41.954320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:41.955979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:42.061902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bbfd3-aff8-4712-9dad-5068da161d46 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:42.090968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:42.137086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:42.137686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:42.137966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:42.138241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:42.139890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:42.256108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:42.257554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e971-0711-4bda-9a64-4632b7e74324 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:42.275153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 670/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:44.511865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:44.512424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:44.512424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:44.513773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 668/1338] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:44.518958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4999-5d6e-40c9-b880-4d74d564eb86 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:44.519333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4999-5d6e-40c9-b880-4d74d564eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:44.519516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4999-5d6e-40c9-b880-4d74d564eb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:44.585942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-218d4999-5d6e-40c9-b880-4d74d564eb86 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:44.587123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-218d4999-5d6e-40c9-b880-4d74d564eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:44.595327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:44.684847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 671/1339] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:44.972635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:44.974212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:45.076653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6973eb-c471-42b3-9678-51d4a6497d92 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:45.149132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:45.149731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:45.150009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:45.150278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:45.152019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:45.187101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:45.260102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:45.261537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2e2ff-a9c5-4061-bb5c-52707216f908 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:45.280163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 669/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:49.185256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:49.185783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:49.186111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:49.187534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 672/1341] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:49.193482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09754-8e67-4544-a183-7372f7e09389 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:49.193997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09754-8e67-4544-a183-7372f7e09389 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:49.194390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09754-8e67-4544-a183-7372f7e09389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:49.263003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1db09754-8e67-4544-a183-7372f7e09389 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:49.264212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1db09754-8e67-4544-a183-7372f7e09389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:49.356753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:49.368780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 670/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:49.654744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:49.656367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:49.762888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6c0a9-c0bd-4e82-92e3-7e6eff5e1ac2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:49.801042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:49.839329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:49.839963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:49.840240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:49.840533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:49.842130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:49.953081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:49.954437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e47016-2f97-4487-b06d-5751e43a0c09 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:49.972335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 673/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:52.557988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.558583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.558583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.560266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 671/1344] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:52.565447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2f4c-57c3-4bd2-8371-efa802d9fd21 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.565838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2f4c-57c3-4bd2-8371-efa802d9fd21 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.566112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a2f4c-57c3-4bd2-8371-efa802d9fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.635781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-bb8a2f4c-57c3-4bd2-8371-efa802d9fd21 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:52.636987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bb8a2f4c-57c3-4bd2-8371-efa802d9fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.655146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:52.735329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 674/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:53.025031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:53.026698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:53.133169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acdcc4-7ec2-4462-b8cc-1a5b03349212 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:53.206730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:53.207336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:53.207617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:53.207890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:53.209479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:53.257061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:53.328768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:53.330226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b13ea-d73f-4d09-8e60-b83cdc5ce9c8 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:53.348981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 672/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:57.270455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:57.271054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:57.271161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:57.273207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 675/1347] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:29:57.278765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7e73-919c-4240-8d69-0d8a7d8571a3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:57.279442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7e73-919c-4240-8d69-0d8a7d8571a3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:57.279442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c7e73-919c-4240-8d69-0d8a7d8571a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:57.367907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-763c7e73-919c-4240-8d69-0d8a7d8571a3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:57.369140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-763c7e73-919c-4240-8d69-0d8a7d8571a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:57.477226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:57.479911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 673/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:57.767264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:57.768884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:57.878202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b78ae-a577-4568-89b2-0601d328cfa3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:57.927255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:57.957167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:57.957768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:57.958045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:57.958314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:57.960063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:58.088305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:58.089775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57269b5-d4cf-49e6-aa9d-12a2053cff79 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:58.113640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 676/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:03.182971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d42b-4a6b-4581-b267-3fe0bb1cf855 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:03.183508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d42b-4a6b-4581-b267-3fe0bb1cf855 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:03.183508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872d42b-4a6b-4581-b267-3fe0bb1cf855 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:03.185160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 674/1350] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:03.192154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541e89-b374-46a4-b9bf-4dc1c19c9489 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:03.200555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541e89-b374-46a4-b9bf-4dc1c19c9489 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:03.200867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541e89-b374-46a4-b9bf-4dc1c19c9489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:03.315292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-18541e89-b374-46a4-b9bf-4dc1c19c9489 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:03.316871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-18541e89-b374-46a4-b9bf-4dc1c19c9489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:03.369418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:03.478788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 677/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:07.651482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dddf59-e53c-4990-aeb7-f13b7403ff9c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:07.652373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dddf59-e53c-4990-aeb7-f13b7403ff9c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:07.652952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dddf59-e53c-4990-aeb7-f13b7403ff9c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:07.654471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 675/1352] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:07.661200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330075bf-49c7-4f68-a445-f497d11c05d4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:07.661665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330075bf-49c7-4f68-a445-f497d11c05d4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:07.661781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330075bf-49c7-4f68-a445-f497d11c05d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:07.749480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-330075bf-49c7-4f68-a445-f497d11c05d4 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:07.751512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-330075bf-49c7-4f68-a445-f497d11c05d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:07.837790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:07.877206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 678/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:09.323942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e9653-3359-4c46-9fb7-2e025385ebd5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:09.324405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e9653-3359-4c46-9fb7-2e025385ebd5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:09.324570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e9653-3359-4c46-9fb7-2e025385ebd5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:09.325957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 676/1354] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:09.331104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc794a9-41ab-4716-b134-49459dbfb58c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:09.331500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc794a9-41ab-4716-b134-49459dbfb58c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:09.331659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc794a9-41ab-4716-b134-49459dbfb58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:09.413586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3dc794a9-41ab-4716-b134-49459dbfb58c None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:09.415630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3dc794a9-41ab-4716-b134-49459dbfb58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:09.530916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 679/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:12.816214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bfbab-ead6-4380-9da6-6037ddfd47df None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:12.816764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bfbab-ead6-4380-9da6-6037ddfd47df None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:12.817131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bfbab-ead6-4380-9da6-6037ddfd47df None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:12.825842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 677/1356] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:12.829105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595c9c-e583-4652-86f1-815e7fc58f68 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:12.829536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595c9c-e583-4652-86f1-815e7fc58f68 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:12.831368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95595c9c-e583-4652-86f1-815e7fc58f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:12.875439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:12.931534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-95595c9c-e583-4652-86f1-815e7fc58f68 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:12.932813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-95595c9c-e583-4652-86f1-815e7fc58f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:13.105448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 680/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:30:13.420878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:13.421401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:13.421903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:13.423823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 678/1358] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 15:30:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:13.467592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea7bcf-3abe-4b33-ad65-db2a12696082 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:13.467985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea7bcf-3abe-4b33-ad65-db2a12696082 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:13.468112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea7bcf-3abe-4b33-ad65-db2a12696082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:13.569464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6fea7bcf-3abe-4b33-ad65-db2a12696082 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:13.571403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6fea7bcf-3abe-4b33-ad65-db2a12696082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:13.632099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:13.715232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 681/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:13.727469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:13.729192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:13.914282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad7216-369f-479f-a3a5-27ccf4fd18ec None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:13.994994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:14.040145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:14.040882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:14.041234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:14.041582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:14.043293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:14.140263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:14.142017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07a3f4-6e1f-4d27-81dd-c44db33e20e3 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:14.167621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 679/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:16.326254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:16.326683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:16.327082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:16.328735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 682/1361] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:16.334170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2167f5-89a0-4f5b-838a-e47307b1b022 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:16.334947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2167f5-89a0-4f5b-838a-e47307b1b022 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:16.335229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2167f5-89a0-4f5b-838a-e47307b1b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:16.409108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ab2167f5-89a0-4f5b-838a-e47307b1b022 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:16.410561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ab2167f5-89a0-4f5b-838a-e47307b1b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:16.453292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:16.528213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 680/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:30:16.824565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:16.826093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:16.994159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a936dc-36ef-44bb-ac14-6a9ea45313de None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:17.004929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:17.124969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:17.125639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:17.126115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:17.126536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:17.128353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:17.232366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:17.234412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67cd4-a507-4b55-9fd4-f8f143d88177 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:17.259539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 683/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:19.556165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01203e1-d389-4080-b7ea-897275f9528b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:19.557041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01203e1-d389-4080-b7ea-897275f9528b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:19.557318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01203e1-d389-4080-b7ea-897275f9528b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:19.558707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 681/1364] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:19.564315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc7b9-32da-4df7-ad58-d73a32cacca1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:19.564750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc7b9-32da-4df7-ad58-d73a32cacca1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:19.564871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dc7b9-32da-4df7-ad58-d73a32cacca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:19.582671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:19.640476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-960dc7b9-32da-4df7-ad58-d73a32cacca1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:19.641678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-960dc7b9-32da-4df7-ad58-d73a32cacca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:19.747152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 684/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:20.044397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:20.044683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:20.044960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:20.046600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 682/1366] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 15:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:20.051816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65d52-a509-4878-be65-ed2656309ac8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:20.052202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65d52-a509-4878-be65-ed2656309ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:20.052738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65d52-a509-4878-be65-ed2656309ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:20.145408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c6b65d52-a509-4878-be65-ed2656309ac8 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:20.159788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c6b65d52-a509-4878-be65-ed2656309ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:20.195965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:20.289494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 685/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:20.297851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:20.299592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:20.491598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a1964-540c-4480-aeed-47552cb55c85 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:20.513745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:20.611748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:20.612397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:20.612639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:20.612768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:20.614351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:20.713794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:20.715486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6242b1-8f81-447d-8fcd-36aed7d5a9f0 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:20.736661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 683/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:21.304756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506274f-6bab-49a7-af23-b97f633d4018 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:21.305419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506274f-6bab-49a7-af23-b97f633d4018 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:21.305419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506274f-6bab-49a7-af23-b97f633d4018 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:21.307105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 686/1369] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:21.312561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b941fb-9fdb-4f11-a034-e0a16432751e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:21.312855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b941fb-9fdb-4f11-a034-e0a16432751e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:21.313137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b941fb-9fdb-4f11-a034-e0a16432751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:21.350378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:21.388701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-87b941fb-9fdb-4f11-a034-e0a16432751e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:21.389999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-87b941fb-9fdb-4f11-a034-e0a16432751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:21.500020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 684/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:22.091942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:22.091942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:22.091942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:22.093797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 687/1371] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 15:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:22.099419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc97e-8fb0-49d2-80d6-4b53a9c6a464 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:22.099881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc97e-8fb0-49d2-80d6-4b53a9c6a464 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:22.100170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc97e-8fb0-49d2-80d6-4b53a9c6a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:22.181166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bcdbc97e-8fb0-49d2-80d6-4b53a9c6a464 None None] MFA Rules not processed for user `6ebbe8c6f1734885ae15e7137ecf8495`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:22.182621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bcdbc97e-8fb0-49d2-80d6-4b53a9c6a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:22.235313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:22.267572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 685/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 15:30:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:22.275792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:22.277434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:22.399401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f5d4-106e-47bd-972e-a089985f6606 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:22.443854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:22.482998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:22.483695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:22.484109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:22.484714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:22.486586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:22.641334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:22.643210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dfc16-8b23-486b-ae7d-68fb052a2c21 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:22.667169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 688/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:22.732087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:22.732265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:22.732715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:22.733929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 686/1374] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:22.739641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee612-f10c-4f65-a0c3-2918af57611f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:22.739919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee612-f10c-4f65-a0c3-2918af57611f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:22.740192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ee612-f10c-4f65-a0c3-2918af57611f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:22.874923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b92ee612-f10c-4f65-a0c3-2918af57611f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:22.876398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b92ee612-f10c-4f65-a0c3-2918af57611f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:22.903705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:22.981870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 689/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:23.276778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:23.278307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:23.448256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e420b9-01a0-4e7f-927e-fdb5c8f0d512 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:23.503973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:23.567397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:23.567920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:23.568198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:23.568470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:23.570080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:23.661525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:23.663172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62549b98-0de9-4662-aa98-672884b2d7ec service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:23.682684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 687/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:25.641078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:25.641598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:25.641598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:25.642903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 690/1377] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:25.648182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90037-4ab6-4f22-b911-3ce20588b816 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:25.648629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90037-4ab6-4f22-b911-3ce20588b816 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:25.648780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e90037-4ab6-4f22-b911-3ce20588b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:25.722012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-47e90037-4ab6-4f22-b911-3ce20588b816 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:25.723389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-47e90037-4ab6-4f22-b911-3ce20588b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:25.731106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:25.862937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 688/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:26.167146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:26.168740 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:26.373775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1f81e-9acf-4d12-bf75-09bf5ac0e9ce None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:26.377946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:26.504927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:26.505568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:26.505847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:26.506150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:26.507929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:26.608802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:26.610602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c5109-cf45-4f07-b29e-47a93f3f37dc service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:26.630073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 691/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:28.644857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:28.645402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:28.645402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:28.646946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 689/1380] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:30:28.653762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905fb1e-0c6b-463a-ba00-2177476f507a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:28.654285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905fb1e-0c6b-463a-ba00-2177476f507a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:28.654285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905fb1e-0c6b-463a-ba00-2177476f507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:28.675008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:28.739962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a905fb1e-0c6b-463a-ba00-2177476f507a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:28.741276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a905fb1e-0c6b-463a-ba00-2177476f507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:28.864429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 692/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:29.163786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:29.165445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:29.368512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:29.382144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea03d4-50cb-4628-a475-66db0864bfbb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:29.533202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:29.533812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:29.538707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:29.538707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:29.538707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:29.647731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:29.649572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829325a-20ee-4790-9ab9-9d166c5a6878 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:29.664325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:29.684600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 690/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:31.702753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:31.703283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:31.703418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:31.705155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 693/1383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:31.715928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a485-08fe-4a65-bca4-f7b325a5c1ad None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:31.715928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a485-08fe-4a65-bca4-f7b325a5c1ad None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:31.715928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8a485-08fe-4a65-bca4-f7b325a5c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:31.791929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0ea8a485-08fe-4a65-bca4-f7b325a5c1ad None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:31.793434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0ea8a485-08fe-4a65-bca4-f7b325a5c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:31.908470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 691/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:30:32.208842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:32.210987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:32.321367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:32.398228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5c29-a948-478c-b84b-1b9791d31930 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:32.544253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:32.544873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:32.545151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:32.545421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:32.547093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:32.611292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:32.662183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:32.663802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03194d7-a8fe-4c5b-b7d1-3f2057c3679b service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:32.686719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 694/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:34.691730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:34.692385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:34.692385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:34.693651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 692/1386] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:34.703871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfefef-444f-4ebb-8730-6c40d3244e9d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:34.704562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfefef-444f-4ebb-8730-6c40d3244e9d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:34.704562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfefef-444f-4ebb-8730-6c40d3244e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:34.787148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-dbcfefef-444f-4ebb-8730-6c40d3244e9d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:34.788356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-dbcfefef-444f-4ebb-8730-6c40d3244e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:34.925603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 695/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:35.223216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:35.224836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:35.321786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:35.408312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc23f39-38c4-4c30-8bf6-c3759f27490f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:35.551874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:35.552829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:35.553215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:35.553215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:35.554874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:35.623572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:35.686663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:35.688754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e34af-7406-4f32-bef3-8a4a9d89188a service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:35.718691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 693/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:30:37.868290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afed8-8f85-4a80-a64a-4142a5cd02c0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:37.868950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afed8-8f85-4a80-a64a-4142a5cd02c0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:37.869058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77afed8-8f85-4a80-a64a-4142a5cd02c0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:37.870939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 696/1389] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:37.876938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46fc7-8902-484c-863a-94a1921b41d2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:37.877420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46fc7-8902-484c-863a-94a1921b41d2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:37.877504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc46fc7-8902-484c-863a-94a1921b41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:37.945429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6fc46fc7-8902-484c-863a-94a1921b41d2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:37.946661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6fc46fc7-8902-484c-863a-94a1921b41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.046739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 694/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:38.258373 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.258676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.258960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.260855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 697/1391] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:38.266012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101cedd-c86f-4cf2-9c78-8241bb896cd8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.266301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101cedd-c86f-4cf2-9c78-8241bb896cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.266579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101cedd-c86f-4cf2-9c78-8241bb896cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.282799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:38.340409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:38.342820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.344459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b101cedd-c86f-4cf2-9c78-8241bb896cd8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:38.345995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b101cedd-c86f-4cf2-9c78-8241bb896cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.399459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:38.487919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 695/1392] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:38.571068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22edc3-694c-4beb-9dbc-736fb09909a7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:38.705967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:38.718798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:38.719435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.719710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.719974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.721555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.790192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517de73-efbd-4fab-8d2b-60f1d752614b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.790553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517de73-efbd-4fab-8d2b-60f1d752614b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.790876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517de73-efbd-4fab-8d2b-60f1d752614b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.792693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 696/1393] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 15:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:30:38.797678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db392-a34c-4800-a659-db8ed229aa09 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.797964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db392-a34c-4800-a659-db8ed229aa09 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.798243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3db392-a34c-4800-a659-db8ed229aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.813477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:38.815105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:38.834203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 698/1394] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:38.879241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0f3db392-a34c-4800-a659-db8ed229aa09 None None] MFA Rules not processed for user `f978768101cd4e478b10b2f919383518`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:38.880488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0f3db392-a34c-4800-a659-db8ed229aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.918803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:38.970391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 697/1395] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:30:38.979189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:38.980739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:39.066952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b384-32f4-48fe-a25f-1ce2127bae5d service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:39.109316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:39.143490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:39.144108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:39.144387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:39.144662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:39.146265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:39.258209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:39.259693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be60a7c-4542-4be4-8855-53c5c21c201a service glance] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:39.277884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 699/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:40.019671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302912b6-dfbe-4ba8-8762-ca648e19d2d5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.020141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302912b6-dfbe-4ba8-8762-ca648e19d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.020244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302912b6-dfbe-4ba8-8762-ca648e19d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.087856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-302912b6-dfbe-4ba8-8762-ca648e19d2d5 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:40.089627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-302912b6-dfbe-4ba8-8762-ca648e19d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:40.180774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 698/1397] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:40.224926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.225431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.225526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.226887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 700/1398] 10.222.0.73 () {54 vars in 943 bytes} [Wed Aug 24 15:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:40.235370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb143-f0fe-41ac-af87-08167e8d309e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.235735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb143-f0fe-41ac-af87-08167e8d309e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.236076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb143-f0fe-41ac-af87-08167e8d309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.294876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:40.315296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-455eb143-f0fe-41ac-af87-08167e8d309e None None] MFA Rules not processed for user `87e124d4d97e46b182279f9b24d2052d`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:40.316646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-455eb143-f0fe-41ac-af87-08167e8d309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:40.425366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 699/1399] 10.222.0.73 () {58 vars in 1040 bytes} [Wed Aug 24 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:40.435456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:40.436988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:40.515762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:40.577956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec07cab-c126-43ee-ae8c-ed82b5c3e33a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:40.681531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:40.682238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.682500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.682800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.684472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:40.744450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:40.770113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:40.771763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69206-c7f1-4bbd-8a02-4f02f1f892a6 service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:40.780775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 701/1400] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:30:40.782802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.783124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.783408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.784771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 700/1401] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:40.790080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b1bb2-450d-4850-a9f9-70fc3423b420 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:40.790368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b1bb2-450d-4850-a9f9-70fc3423b420 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:40.790681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b1bb2-450d-4850-a9f9-70fc3423b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:40.861846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-726b1bb2-450d-4850-a9f9-70fc3423b420 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:40.863078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-726b1bb2-450d-4850-a9f9-70fc3423b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:40.984728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 702/1402] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:41.275395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:41.277866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:41.397347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:41.445975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb6f49-61c2-467a-98d4-0d31a9bd1180 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:41.580422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:41.580946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:41.581235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:41.581516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:41.583230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:41.672433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:41.693874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:41.695763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a132a5-0c6c-45e9-bb6a-3cef274d3283 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:41.723124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 701/1403] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:43.743506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:43.744071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:43.744071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:43.745881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 703/1404] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:43.751270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b6673-8261-4d02-819e-a791bf3f95e0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:43.751740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b6673-8261-4d02-819e-a791bf3f95e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:43.751839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b6673-8261-4d02-819e-a791bf3f95e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:43.825123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-638b6673-8261-4d02-819e-a791bf3f95e0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:43.830117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-638b6673-8261-4d02-819e-a791bf3f95e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:43.949159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 702/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:44.248969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:44.250496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:44.306094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:44.412855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e63c-a684-482e-a532-148c4d30bdd5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:44.536627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:44.537229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:44.537506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:44.537779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:44.539401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:44.552342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:44.638200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:44.640029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771737ad-f73f-4b75-8c25-f5fc745af7de service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:44.660358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 704/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:46.675451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:46.676023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:46.676023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:46.677360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 703/1407] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:46.683356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed80ed-7ee1-4361-943a-3d5112c24a4d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:46.683999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed80ed-7ee1-4361-943a-3d5112c24a4d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:46.683999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed80ed-7ee1-4361-943a-3d5112c24a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:46.757786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-03ed80ed-7ee1-4361-943a-3d5112c24a4d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:46.759059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-03ed80ed-7ee1-4361-943a-3d5112c24a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:46.864330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:46.881187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 705/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:47.185957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:47.188141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:47.250997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:47.375476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c006d2-3e32-4408-bbc7-bc84b377e328 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:47.505015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:47.505623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:47.505899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:47.506167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:47.507821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:47.517045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:47.609104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:47.610969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a770af9-90f3-4883-a593-c84dd01b0b13 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:47.630548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 704/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:56.879169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:57.628133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=41 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:03.113598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f889bf-7773-4810-b4c2-0384de592282 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:03.114471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f889bf-7773-4810-b4c2-0384de592282 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:03.114471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f889bf-7773-4810-b4c2-0384de592282 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:03.115792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 706/1410] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 15:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:03.120976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae873a-c7ec-455b-bb9d-bc72840607f8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:03.121500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae873a-c7ec-455b-bb9d-bc72840607f8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:03.121607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae873a-c7ec-455b-bb9d-bc72840607f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:03.196703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2fae873a-c7ec-455b-bb9d-bc72840607f8 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:03.197905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2fae873a-c7ec-455b-bb9d-bc72840607f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:03.302109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 705/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:03.319158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b655e-d05b-4b43-ac7a-7356f74a7c46 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:03.320203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b655e-d05b-4b43-ac7a-7356f74a7c46 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:03.320203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b655e-d05b-4b43-ac7a-7356f74a7c46 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:03.321217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 707/1412] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 15:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:03.326784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8bb3d-db22-4f7c-a820-67f7cd3e60d3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:03.327080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8bb3d-db22-4f7c-a820-67f7cd3e60d3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:03.327359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8bb3d-db22-4f7c-a820-67f7cd3e60d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:03.401523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:03.405004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-22e8bb3d-db22-4f7c-a820-67f7cd3e60d3 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:03.406667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22e8bb3d-db22-4f7c-a820-67f7cd3e60d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:03.523519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 706/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:05.941416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:05.942086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:05.942370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:05.943907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:05.949678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65555-48f7-4583-bb1d-47f98cba59f3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:05.950165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65555-48f7-4583-bb1d-47f98cba59f3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:05.950509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65555-48f7-4583-bb1d-47f98cba59f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:06.020944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1df65555-48f7-4583-bb1d-47f98cba59f3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:06.022162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1df65555-48f7-4583-bb1d-47f98cba59f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:06.084685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:06.126931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:06.419568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:06.421224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:06.569771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52385536-53e0-4478-b0a8-253b1b203bd8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:06.642724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:06.683142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:06.683746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:06.684022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:06.684294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:06.685894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:06.776450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:06.778777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8485-0627-49ae-80e1-dc174bcee8c8 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:06.796594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 709/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:07.685870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:07.686414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:07.686414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:07.687781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:07.692642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe92b4-18ab-418a-afb0-70a3bcbf1ae7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:07.693043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe92b4-18ab-418a-afb0-70a3bcbf1ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:07.693193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe92b4-18ab-418a-afb0-70a3bcbf1ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:07.774126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-aebe92b4-18ab-418a-afb0-70a3bcbf1ae7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:07.775359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-aebe92b4-18ab-418a-afb0-70a3bcbf1ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:07.802439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:07.872573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:08.021937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:08.023500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:08.168500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37734c43-d97e-4d0d-909b-f50d8151ad1c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:08.208277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:08.282751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:08.283287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:08.283565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:08.283837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:08.285421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:08.374160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:08.375772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cedd3-5cad-4cc5-9510-28bbe5de6195 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:08.393889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 709/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:09.284830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:09.285379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:09.285379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:09.286737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:09.291729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314c841-165e-476d-9d54-2ee0640d1c90 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:09.292151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314c841-165e-476d-9d54-2ee0640d1c90 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:09.292286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314c841-165e-476d-9d54-2ee0640d1c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:09.347556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:09.369609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4314c841-165e-476d-9d54-2ee0640d1c90 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:09.370874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4314c841-165e-476d-9d54-2ee0640d1c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:09.470304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:09.620736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:09.622761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:09.769194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd8bbb-ade8-4228-b606-c1c54deaee26 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:09.781239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:09.882295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:09.882846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:09.883130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:09.883404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:09.885018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:09.973633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:09.975270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef825c1-db68-4a6f-bc9e-f7b4871ca118 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:09.994385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 712/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:10.862342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:10.862945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:10.862945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:10.864331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:10.870465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7753f4-491b-4235-a9de-31d2fd365b1e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:10.871001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7753f4-491b-4235-a9de-31d2fd365b1e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:10.871290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7753f4-491b-4235-a9de-31d2fd365b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:10.893472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:10.949658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0f7753f4-491b-4235-a9de-31d2fd365b1e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:10.951595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0f7753f4-491b-4235-a9de-31d2fd365b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:11.048101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:11.199133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:11.201615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:11.311692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:11.348220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ddb80-072e-4c70-a232-5ebde40ec112 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:11.463181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:11.463704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:11.463984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:11.464258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:11.465923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:11.547591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:11.561881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:11.563532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae198b1-0403-470e-b0c0-58f8fa802d00 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:11.581307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 712/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:12.448308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28445-e8af-41cd-b556-7e921470518f None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:12.448829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28445-e8af-41cd-b556-7e921470518f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:12.448829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a28445-e8af-41cd-b556-7e921470518f None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:12.450165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:12.455900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d70bb0-2d8f-4966-8b6d-b702315a4b2f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:12.456317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d70bb0-2d8f-4966-8b6d-b702315a4b2f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:12.456455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d70bb0-2d8f-4966-8b6d-b702315a4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:12.532127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f6d70bb0-2d8f-4966-8b6d-b702315a4b2f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:12.533358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f6d70bb0-2d8f-4966-8b6d-b702315a4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:12.630536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:12.781349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a28445-e8af-41cd-b556-7e921470518f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:12.783033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-39a28445-e8af-41cd-b556-7e921470518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:12.866991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:12.933019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a28445-e8af-41cd-b556-7e921470518f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:13.047307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:13.047828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:13.048103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:13.048369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:13.049952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:13.096636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:13.138154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:13.139786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ed89e-8303-420f-a071-cba1c069ddff service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:13.158051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 715/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:13.964285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:13.964877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:13.964877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:13.966211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:13.971330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef01f-977d-4c0c-85c0-f7e1936a53a9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:13.971729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef01f-977d-4c0c-85c0-f7e1936a53a9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:13.971892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef01f-977d-4c0c-85c0-f7e1936a53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:14.056165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1acef01f-977d-4c0c-85c0-f7e1936a53a9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:14.057387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1acef01f-977d-4c0c-85c0-f7e1936a53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:14.157297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:14.306902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:14.308433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:14.357579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:14.456395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634b6d7-db25-49ca-b88b-123e32d2ec13 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:14.574453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:14.574996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:14.575273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:14.575600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:14.577289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:14.584208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:14.666298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:14.667940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee449ab9-2d15-45ac-9252-cbdb76fdab3a service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:14.686530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 715/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:15.522452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:15.523076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:15.523076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:15.524394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:15.529453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bae0-7c21-45ce-845d-6487a624972d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:15.529877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bae0-7c21-45ce-845d-6487a624972d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:15.529997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bae0-7c21-45ce-845d-6487a624972d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:15.607267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5208bae0-7c21-45ce-845d-6487a624972d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:15.608406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5208bae0-7c21-45ce-845d-6487a624972d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:15.675495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:15.708807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:15.862674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:15.864128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:15.882383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:16.013829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50606cfd-e286-4321-a44c-fe646503db02 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:16.119265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:16.145356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:16.145957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:16.146238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:16.146513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:16.148165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:16.241474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:16.243128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449d0d-6b0f-4e7b-8ab7-9f4db96253f5 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:16.260897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 718/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:17.080906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:17.081503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:17.081503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:17.082881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:17.088057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9307d1-86bc-47ac-b6d9-57fe8ce3b116 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:17.088519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9307d1-86bc-47ac-b6d9-57fe8ce3b116 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:17.088674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9307d1-86bc-47ac-b6d9-57fe8ce3b116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:17.165223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5c9307d1-86bc-47ac-b6d9-57fe8ce3b116 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:17.166432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5c9307d1-86bc-47ac-b6d9-57fe8ce3b116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:17.212754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:17.271051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:17.424763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:17.426318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:17.572420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6465f-ff92-4418-9ec7-1091af6c9cc6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:17.627340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:17.686476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:17.687114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:17.687392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:17.687661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:17.689249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:17.778099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:17.779774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60e466-f228-4027-bedf-6b8729ec39ee service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:17.798426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 718/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:18.617382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:18.617961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:18.617961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:18.619322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:18.624326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a23c1-b2d8-4606-9d49-c4de492b80a6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:18.624815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a23c1-b2d8-4606-9d49-c4de492b80a6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:18.624927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a23c1-b2d8-4606-9d49-c4de492b80a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:18.700490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-305a23c1-b2d8-4606-9d49-c4de492b80a6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:18.701762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-305a23c1-b2d8-4606-9d49-c4de492b80a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:18.703837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:18.797720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:31:18.957823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:18.959384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:19.106493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22240a5f-6081-4f25-8d8e-2ee21653ab5c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:19.133897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:19.223056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:19.223558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:19.223832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:19.224099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:19.225692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:19.317204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:19.318851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f11a4-4346-40f9-8b6f-657395028f8f service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:19.337328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 721/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:21.165894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:21.166434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:21.166434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:21.167835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:21.173381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7527-e3bc-4c5e-acfd-d8e47f5a70a8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:21.173792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7527-e3bc-4c5e-acfd-d8e47f5a70a8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:21.173933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7527-e3bc-4c5e-acfd-d8e47f5a70a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:21.214526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:21.240276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-058b7527-e3bc-4c5e-acfd-d8e47f5a70a8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:21.241739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-058b7527-e3bc-4c5e-acfd-d8e47f5a70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:21.340186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:21.628136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:21.629674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:21.733312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd948c3-f5cc-43dd-ac52-99a5cef5c76f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:21.765525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:21.807364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:21.807969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:21.808250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:21.808522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:21.810132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:21.918120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:21.919529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4445-a8c5-48bb-b2c3-be4c79035adc service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:21.938940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 721/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:23.973262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:23.973837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:23.973960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:23.975245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:23.980367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596add6-f631-4171-ba0c-d99c20ce8532 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:23.980762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596add6-f631-4171-ba0c-d99c20ce8532 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:23.980937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596add6-f631-4171-ba0c-d99c20ce8532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:24.048444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c596add6-f631-4171-ba0c-d99c20ce8532 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:24.049650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c596add6-f631-4171-ba0c-d99c20ce8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:24.062331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:24.159974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:24.449416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:24.450980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:24.554880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec33d3-3ceb-44a4-80db-dbfe68a2cdff None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:24.567520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:24.629706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:24.630204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:24.630480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:24.630792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:24.632373 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:24.742849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:24.744431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e078fc7-ae32-4f6d-8ac1-7359743ac8c4 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:24.762825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 724/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:26.968412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:26.968977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:26.968977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:26.970352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:31:26.975822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd1a6-1ea7-446a-a4d7-022fbe82c414 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:26.976257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd1a6-1ea7-446a-a4d7-022fbe82c414 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:26.976461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd1a6-1ea7-446a-a4d7-022fbe82c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:27.024898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:27.043358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-690cd1a6-1ea7-446a-a4d7-022fbe82c414 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:27.044597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-690cd1a6-1ea7-446a-a4d7-022fbe82c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:27.142152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:27.433854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:27.435435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:27.538846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84427b-bea0-4995-ac8f-7c127c85d41a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:27.580198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:27.614473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:27.615112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:27.615394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:27.615666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:27.617535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:27.725292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:27.726769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937d486-b5e6-49dc-9c7c-d360ae350618 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:27.745325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 724/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:29.828080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:29.828649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:29.828649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:29.829960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:29.835500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b931e-f0e1-4e43-ab79-067d3d0e852f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:29.835944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b931e-f0e1-4e43-ab79-067d3d0e852f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:29.836054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b931e-f0e1-4e43-ab79-067d3d0e852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:29.902331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b19b931e-f0e1-4e43-ab79-067d3d0e852f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:29.903583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b19b931e-f0e1-4e43-ab79-067d3d0e852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:29.925186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:29.999970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:30.290403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:30.291996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:30.396197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4318ccf-cd34-4445-b272-338ac6856c12 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:30.415518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:30.472640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:30.473276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:30.473558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:30.473830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:30.475455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:30.585021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:30.586452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42b0a-1a52-43a4-9268-6891550a8d7d service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:30.603978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 727/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:32.696524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:32.697100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:32.697100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:32.698430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:32.704438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36553a1-38c0-4414-a754-a7f8f0b24629 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:32.704835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36553a1-38c0-4414-a754-a7f8f0b24629 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:32.705003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36553a1-38c0-4414-a754-a7f8f0b24629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:32.769065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:32.771727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a36553a1-38c0-4414-a754-a7f8f0b24629 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:32.773513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a36553a1-38c0-4414-a754-a7f8f0b24629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:32.875428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:33.166739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:33.168279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:33.272673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37058057-667c-46c4-947a-7fa97b460ad8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:33.321492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:33.345977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:33.346579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:33.346887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:33.347172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:33.348800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:33.458346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:33.459752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2846b-c9da-43c0-bde9-98647b313151 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:33.478103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 727/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:35.455141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:35.455699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:35.455699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:35.457330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:35.462758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b980-64a3-4081-8524-f25f3346b800 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:35.463206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b980-64a3-4081-8524-f25f3346b800 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:35.463326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b980-64a3-4081-8524-f25f3346b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:35.528797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8d09b980-64a3-4081-8524-f25f3346b800 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:35.529994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d09b980-64a3-4081-8524-f25f3346b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:35.559582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:35.628582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:35.916477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:35.918043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:36.021762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d18868-6c8a-4ac8-8514-6d75a2f28d9d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:36.049859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:36.097562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:36.098162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:36.098436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:36.098729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:36.100331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:36.213509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:36.214891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2408da3-2e95-46e2-b3b1-ceb7904c8e50 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:36.235220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 730/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:38.309152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:38.309740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:38.309740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:38.311112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:38.316699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bfbc2-30ca-416e-99bd-4c705e274078 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:38.317210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bfbc2-30ca-416e-99bd-4c705e274078 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:38.317295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bfbc2-30ca-416e-99bd-4c705e274078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:38.386265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-368bfbc2-30ca-416e-99bd-4c705e274078 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:38.387402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-368bfbc2-30ca-416e-99bd-4c705e274078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:38.396362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:38.603703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:38.888135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:38.889776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:38.995863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9459a-42fc-4f8a-837a-70bb71bd802b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:39.052687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:39.071487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:39.072094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:39.072368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:39.072639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:39.074245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:39.182366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:39.183747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11006180-2b5c-4583-9641-d24d80b8ee86 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:39.201912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 730/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:41.251029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:41.251575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:41.251697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:41.252953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:41.258237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888fc04-ab79-49a0-ae09-2492cb0e8c2d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:41.258723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888fc04-ab79-49a0-ae09-2492cb0e8c2d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:41.258837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888fc04-ab79-49a0-ae09-2492cb0e8c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:41.324361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5888fc04-ab79-49a0-ae09-2492cb0e8c2d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:41.325575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5888fc04-ab79-49a0-ae09-2492cb0e8c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:41.364361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:41.423049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:41.705188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:41.706738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:41.807822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d68eb-9ca9-4b3b-b8ca-f7a787b1ae7e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:41.845281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:41.882203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:41.882828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:41.883105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:41.883378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:41.884967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:41.992520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:41.994116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372de52-14df-4872-97a5-4136efd0f2bd service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:42.012304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 733/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:43.965974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:43.966675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:43.966675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:43.967927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:43.974069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85358f4c-23fb-4227-9cdd-68ec3d51ac71 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:43.974479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85358f4c-23fb-4227-9cdd-68ec3d51ac71 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:43.974619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85358f4c-23fb-4227-9cdd-68ec3d51ac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:44.041048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-85358f4c-23fb-4227-9cdd-68ec3d51ac71 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:44.042279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-85358f4c-23fb-4227-9cdd-68ec3d51ac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:44.060739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:44.139876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:44.435383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:44.436948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:44.540813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23390994-24e9-4986-8f4b-8df36cc20656 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:44.608752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:44.617612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:44.618232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:44.618495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:44.618799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:44.620651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:44.737242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:44.738706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfd340-5665-41f3-9b04-6220c85c6adc service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:44.756715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 733/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:46.720347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.720949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.720949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.722244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:31:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:46.727714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded62e2-b294-43ee-922a-bcf0ee851392 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.728147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded62e2-b294-43ee-922a-bcf0ee851392 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.728266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded62e2-b294-43ee-922a-bcf0ee851392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.795795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-aded62e2-b294-43ee-922a-bcf0ee851392 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:46.797066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-aded62e2-b294-43ee-922a-bcf0ee851392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.844115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:46.893827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:47.177000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:47.178932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:47.282511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc19d0-d921-420b-ad4d-fd068a5ea11e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:47.330071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:47.355724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:47.356333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:47.356612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:47.356885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:47.358508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:47.466943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:47.468377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff17a2-02be-4c84-8662-544c476bb1fb service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:47.487014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 736/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:56.892674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=22 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:57.484108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=66 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:07.151428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56a26-f650-4962-98b6-fa26dc65d200 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:07.151998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56a26-f650-4962-98b6-fa26dc65d200 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:07.151998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e56a26-f650-4962-98b6-fa26dc65d200 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:07.153334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 735/1471] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:07.158972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5018f5-1f11-4982-867a-3946286f95b0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:07.159660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5018f5-1f11-4982-867a-3946286f95b0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:07.159660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5018f5-1f11-4982-867a-3946286f95b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:07.223924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-bb5018f5-1f11-4982-867a-3946286f95b0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:07.225122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bb5018f5-1f11-4982-867a-3946286f95b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:07.320206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 737/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:07.618231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:07.618516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:07.618832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:07.620179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 736/1473] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 15:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:07.625425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1c7f-cb13-4b08-816e-5513477300d7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:07.625711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1c7f-cb13-4b08-816e-5513477300d7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:07.625986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d1c7f-cb13-4b08-816e-5513477300d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:07.701276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-da6d1c7f-cb13-4b08-816e-5513477300d7 None None] MFA Rules not processed for user `f978768101cd4e478b10b2f919383518`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:07.702466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-da6d1c7f-cb13-4b08-816e-5513477300d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:07.723810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:07.779905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 738/1474] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:07.788526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:07.790120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:07.899645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f6c90-34d8-451f-90ec-bbf83260f7f0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:07.973045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:07.973651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:07.973960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:07.974232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:07.975874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:08.039608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:08.097109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:08.098556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca792d-7db6-4d82-80ed-f6b3f53ceb65 service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:08.118407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 737/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:10.175737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:10.176241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:10.176340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:10.177648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 739/1476] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:10.183212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08d9c-5cc4-42bd-9ebd-651ab24fd320 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:10.183660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08d9c-5cc4-42bd-9ebd-651ab24fd320 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:10.183756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08d9c-5cc4-42bd-9ebd-651ab24fd320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:10.256006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d0d08d9c-5cc4-42bd-9ebd-651ab24fd320 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:10.257229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d0d08d9c-5cc4-42bd-9ebd-651ab24fd320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:10.361480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 738/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:10.641476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:10.643134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:10.776255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1586-c8a1-4785-97e7-335190c7a9b9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:10.826502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:10.875111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:10.875720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:10.876000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:10.876292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:10.876920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:10.879394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:32:10.879394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:32:10.879394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:32:10.880546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3884853-2418-4953-bbe4-5a4075c7d58b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:10.900597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 740/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 15:32:10 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:32:12.926430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:12.926987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:12.926987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:12.928396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 739/1479] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:12.934197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78804-339b-47ff-a40b-e594b508a2eb None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:12.934691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78804-339b-47ff-a40b-e594b508a2eb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:12.934793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78804-339b-47ff-a40b-e594b508a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:13.010360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0cd78804-339b-47ff-a40b-e594b508a2eb None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:13.012394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0cd78804-339b-47ff-a40b-e594b508a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:13.115972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 741/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:13.418621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:13.420373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:13.432363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:13.581650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85141865-6e0f-44a5-9c8e-e8384007b4a6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:13.666008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:13.700474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:13.701095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:13.701382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:13.701654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:13.703316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:13.797644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:13.799376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b39b1-ec66-410d-8759-b2f96068d186 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:13.817925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 740/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:16.002915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:16.003593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:16.003593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:16.004892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 742/1482] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:16.010120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5ea8-743f-40d3-b343-53bca1cd4bb4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:16.010582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5ea8-743f-40d3-b343-53bca1cd4bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:16.010733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5ea8-743f-40d3-b343-53bca1cd4bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:16.077199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-86ae5ea8-743f-40d3-b343-53bca1cd4bb4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:16.078410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-86ae5ea8-743f-40d3-b343-53bca1cd4bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:16.124525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:16.182880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 741/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:16.473462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:16.475008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:16.498968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:16.582213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3800f-3bdd-4fef-95a5-40df5fd335b0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:16.654854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:16.655458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:16.655739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:16.656016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:16.657647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:16.735396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:16.773303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:16.774825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed782244-e5d1-4742-8a0b-06ac8435ad47 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:16.793807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 743/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:32:18.770060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:18.770616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:18.770616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:18.772926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 742/1485] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:18.778261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52baed-f2ef-43fc-bbbf-cef332926666 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:18.778736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52baed-f2ef-43fc-bbbf-cef332926666 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:18.778871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52baed-f2ef-43fc-bbbf-cef332926666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:18.860606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ba52baed-f2ef-43fc-bbbf-cef332926666 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:18.861864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ba52baed-f2ef-43fc-bbbf-cef332926666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:18.959726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 744/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:19.246960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:19.248795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:19.352401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49ca39-fffd-49c2-8f5c-c7ff973856ef None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:19.423658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:19.426163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:19.426791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:19.427300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:19.427786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:19.429619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:19.540094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:19.541535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be693f9c-aee5-4789-bcb7-c1979ecf3cd7 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:19.559351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 743/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:21.586400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:21.586959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:21.586959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:21.588271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 745/1488] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:21.593432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec3ce-82f0-46a8-9395-27fb336c43f3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:21.593868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec3ce-82f0-46a8-9395-27fb336c43f3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:21.593989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec3ce-82f0-46a8-9395-27fb336c43f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:21.659356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-14bec3ce-82f0-46a8-9395-27fb336c43f3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:21.661230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-14bec3ce-82f0-46a8-9395-27fb336c43f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:21.712504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:21.758228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 744/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:22.070963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:22.072509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:22.109666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:22.178837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc57a6c-3066-4747-a7d9-f3327dc36d11 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:22.256534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:22.257139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:22.257422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:22.257698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:22.259352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:22.339393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:22.368286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:22.369730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa0c71-513e-446c-9d49-600cc0a5d69b service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:22.388284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 746/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:24.464834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:24.465377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:24.465377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:24.466691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 745/1491] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 15:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:32:24.472121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fb44-72c7-4927-b65a-3c8ed5e07ce7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:24.472575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fb44-72c7-4927-b65a-3c8ed5e07ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:24.472713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0fb44-72c7-4927-b65a-3c8ed5e07ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:24.539260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b9c0fb44-72c7-4927-b65a-3c8ed5e07ce7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:24.540469 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b9c0fb44-72c7-4927-b65a-3c8ed5e07ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:24.637668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 747/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:24.931272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:24.932861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:25.037296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0df5-42e3-4d81-8eaa-dddbba672753 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:25.126217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:25.126855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:25.127138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:25.127411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:25.129614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:25.138866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:25.237522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:25.239060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbac4d-a7e9-4ffb-91b8-a294bc21a8db service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:25.257357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 746/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:34.635925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=82 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:35.254506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=50 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:37.582124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:37.582743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:37.582743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:37.659423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:33:37.660641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:37.758015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 748/1494] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:33:37.767714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:37.768300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:37.768300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:37.847346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:33:37.848499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:37.944495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 747/1495] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:33:37.949800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:37.951044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:37.962313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c411-5e28-47ab-b8e8-3cbff358de5d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.023429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:38.056828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.057425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.057700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.057967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.058822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:33:38.059126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.060125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.060125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:33:38.060125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:33:38.061003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.071815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 749/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:33:38.081036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:38.090856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2b33a-87b2-4864-ad4c-6f36a5f5a95d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.153071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:38.188054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.188657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.188935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.189203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.189816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.191623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.191623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:33:38.191623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:33:38.192852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.250835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 748/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:33:38 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:33:38.255089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:38.256609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:38.290295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6981a-6e49-4dd7-9afb-25649d49626c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.389066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.389661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.389940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.390213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.390867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.391913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.391913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:33:38.391913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:33:38.393216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.436817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 750/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:33:38 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:33:38.440912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:38.442490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:38.503881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed57a9-b58e-4017-8162-7887420ca181 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.588704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:38.600156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.600761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.601061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.601335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.601977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.603009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.603009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:33:38.603009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:33:38.603901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.616469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 749/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:33:38.620617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:38.622836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:38.633989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36140892-f91e-4035-97d5-7dfef1a07fb5 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.659627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:38.731938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.732459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.732731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.732997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.733634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.734632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.734632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:33:38.734632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:33:38.735542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5508a6-2bf7-4602-9c82-68a970afbd88 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.748751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 751/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:33:38.753179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:38.754785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:38.796312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0392b-eef4-4fe6-918e-3541e1ec5499 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:38.894612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:38.895302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.895584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.895864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] PATH_INFO: `/v3/projects/4af32f683dbd42cf9ec97e543d9793d1/users/c9318be24f9d4b469565e6a95891c0fa/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:38.917518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af32f683dbd42cf9ec97e543d9793d1, user_id=c9318be24f9d4b469565e6a95891c0fa, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:38.919121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:38.919121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:33:38.919121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:33:38.921172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22186622-ef4b-4db3-aee7-a76200b548c0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:38.946625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 750/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:33:38 2022] PUT /identity/v3/projects/4af32f683dbd42cf9ec97e543d9793d1/users/c9318be24f9d4b469565e6a95891c0fa/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:33:38.958666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f3175-b9e6-438a-918d-5d66840e4984 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:38.958966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f3175-b9e6-438a-918d-5d66840e4984 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:38.959247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f3175-b9e6-438a-918d-5d66840e4984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.019223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7d3f3175-b9e6-438a-918d-5d66840e4984 None None] MFA Rules not processed for user `c9318be24f9d4b469565e6a95891c0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:33:39.020422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d3f3175-b9e6-438a-918d-5d66840e4984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:39.097371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 752/1502] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:33:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:33:39.111178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.111465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.111742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.113152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 751/1503] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 15:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:33:39.119383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fafc5-08a2-4c78-a2d7-e59db7042777 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.119668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fafc5-08a2-4c78-a2d7-e59db7042777 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.119973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fafc5-08a2-4c78-a2d7-e59db7042777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.153895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:39.199103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ac0fafc5-08a2-4c78-a2d7-e59db7042777 None None] MFA Rules not processed for user `6ebbe8c6f1734885ae15e7137ecf8495`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:33:39.200332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ac0fafc5-08a2-4c78-a2d7-e59db7042777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:39.284898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 753/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 15:33:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:33:39.292738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:39.295064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:39.345844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:39.403150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69ebb1-73f4-4603-bdda-eae525aa413f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:39.478041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:39.478671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.478959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.479236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.480912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:39.578853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:39.580206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:39.583713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:39.599067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 752/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:33:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:33:39.643764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe554b-55d0-44f7-8e6a-a37fe4a29462 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.644208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe554b-55d0-44f7-8e6a-a37fe4a29462 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.644330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe554b-55d0-44f7-8e6a-a37fe4a29462 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.645592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 754/1506] 10.222.0.73 () {52 vars in 830 bytes} [Wed Aug 24 15:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:33:39.649234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:39.650482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:39.749622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b541a-b87d-4375-8d7d-f02e615443dd service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:39.844521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:39.845222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.845324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.845569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] PATH_INFO: `/v3/users/c9318be24f9d4b469565e6a95891c0fa` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:39.857507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9318be24f9d4b469565e6a95891c0fa)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:39.858524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:39.858524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:33:39.858524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:33:39.859773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:39.869418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:39.925104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9318be24f9d4b469565e6a95891c0fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:39.925555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9318be24f9d4b469565e6a95891c0fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:39.940738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9318be24f9d4b469565e6a95891c0fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:39.941298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9318be24f9d4b469565e6a95891c0fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:39.951026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9318be24f9d4b469565e6a95891c0fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:39.980577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 753/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:33:39 2022] DELETE /identity/v3/users/c9318be24f9d4b469565e6a95891c0fa => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:33:39.984999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:39.985315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:39.985595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:40.041875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:33:40.043328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:40.118603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:40.131230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 755/1508] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:33:40.141711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:40.142990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:40.363450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dcaba-5a11-4e32-ae10-24874b5eb1cc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:40.417255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:40.440054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:40.440668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:40.440948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:40.441219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:40.442862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:40.551426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:40.552900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3dd845-a8e7-451a-990f-6d7d020ea35a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:40.570679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 754/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:33:40.935159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:40.936392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:40.947904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb87b-d936-41a2-9a24-53d8aa29133e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:41.047038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:41.047669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:41.047947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:41.048217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] PATH_INFO: `/v3/projects/4af32f683dbd42cf9ec97e543d9793d1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:41.058002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af32f683dbd42cf9ec97e543d9793d1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:41.059111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:33:41.059111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:33:41.059111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:33:41.061071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:41.117992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af32f683dbd42cf9ec97e543d9793d1', 'request_id': 'req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:41.124528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af32f683dbd42cf9ec97e543d9793d1', 'request_id': 'req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:41.137622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af32f683dbd42cf9ec97e543d9793d1', 'request_id': 'req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:41.163577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] The token cache is being invalidate because project 4af32f683dbd42cf9ec97e543d9793d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:33:41.163865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:41.164379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4e8de6e-6b66-4a46-a974-af4bb9bdfd03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:33:41.165676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 756/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:33:40 2022] DELETE /identity/v3/projects/4af32f683dbd42cf9ec97e543d9793d1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:33:50.568417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=62 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:51.161642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:13.056598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.057178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.057278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.058582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:13.063596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda131-c6de-4f85-b7c7-68da2be935c3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.064048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda131-c6de-4f85-b7c7-68da2be935c3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.064144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdda131-c6de-4f85-b7c7-68da2be935c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.143516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-efdda131-c6de-4f85-b7c7-68da2be935c3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:13.144711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-efdda131-c6de-4f85-b7c7-68da2be935c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:13.248013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:13.403703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:13.405957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:13.529440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdb179-1b23-43ef-86b4-fb6a4e64e58e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:13.535023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:13.535627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.535903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.536173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.544688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:13.546272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fb29b-3343-4e1b-8113-950aaed0674d service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:13.555472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 756/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:35:14.397908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1692-2599-4cac-aec7-2100195e06c7 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:14.398462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1692-2599-4cac-aec7-2100195e06c7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:14.398462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f1692-2599-4cac-aec7-2100195e06c7 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:14.399838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:14.405074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990e8de-2ade-4596-8536-d3e1f7610dbe None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:14.405531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990e8de-2ade-4596-8536-d3e1f7610dbe None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:14.405699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990e8de-2ade-4596-8536-d3e1f7610dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:14.425132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f990e8de-2ade-4596-8536-d3e1f7610dbe None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:14.426391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f990e8de-2ade-4596-8536-d3e1f7610dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:14.435292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:35:14.587293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1692-2599-4cac-aec7-2100195e06c7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:14.602261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f1692-2599-4cac-aec7-2100195e06c7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:14.608416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:14.609025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:14.609304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:14.609573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:14.618110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:14.619952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9750a43-25d8-4a03-b771-66c7808996a3 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:14.629172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 759/1516] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:15.178064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6775cf-c0d3-4cb6-883d-321a0e3b0b6c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:15.178347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6775cf-c0d3-4cb6-883d-321a0e3b0b6c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:15.178618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6775cf-c0d3-4cb6-883d-321a0e3b0b6c None None] PATH_INFO: `/v3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:15.179930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 758/1517] 10.222.0.18 () {54 vars in 925 bytes} [Wed Aug 24 15:35:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:35:15.210185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dbaeb-815d-43fa-9f41-4d12341cf9ab None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:15.210475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dbaeb-815d-43fa-9f41-4d12341cf9ab None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:15.210796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dbaeb-815d-43fa-9f41-4d12341cf9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:15.273277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5d3dbaeb-815d-43fa-9f41-4d12341cf9ab None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:15.274551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5d3dbaeb-815d-43fa-9f41-4d12341cf9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:15.293515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:15.350958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 760/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 24 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:19.216354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817710-04c2-4d19-9a57-156a688080e0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:19.216911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817710-04c2-4d19-9a57-156a688080e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:19.216911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817710-04c2-4d19-9a57-156a688080e0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:19.218183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 759/1519] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 15:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:19.223151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237385-2d22-4b33-8a0a-91326ff2ce43 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:19.223621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237385-2d22-4b33-8a0a-91326ff2ce43 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:19.223728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237385-2d22-4b33-8a0a-91326ff2ce43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:19.243756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-91237385-2d22-4b33-8a0a-91326ff2ce43 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:19.244944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-91237385-2d22-4b33-8a0a-91326ff2ce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:19.253457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 761/1520] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:19.261121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a817710-04c2-4d19-9a57-156a688080e0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:19.275085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a817710-04c2-4d19-9a57-156a688080e0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:19.280111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:19.280690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:19.280969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:19.281239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:19.290102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:19.291879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07730e-deb6-4d5d-9cc2-3a430b62da85 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:19.301322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 760/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:25.854073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b86-4cb3-46a8-b0fe-a06e9393013f None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:25.854688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b86-4cb3-46a8-b0fe-a06e9393013f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:25.854688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63b86-4cb3-46a8-b0fe-a06e9393013f None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:25.855982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:25.860885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8cb73-56ad-42a2-b628-0ac20570a9e4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:25.861315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8cb73-56ad-42a2-b628-0ac20570a9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:25.861427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8cb73-56ad-42a2-b628-0ac20570a9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:25.883577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9bb8cb73-56ad-42a2-b628-0ac20570a9e4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:25.884800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9bb8cb73-56ad-42a2-b628-0ac20570a9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:25.892789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:26.048372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63b86-4cb3-46a8-b0fe-a06e9393013f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:26.062763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63b86-4cb3-46a8-b0fe-a06e9393013f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:26.068172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:26.068762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:26.069113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:26.069595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:26.077446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:26.079062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa26fd-f73f-45f1-bcbe-99f09c31aa76 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:26.089319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 763/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:26.930555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b17a6-aa2f-4f95-90dc-19c210e40f1e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:26.931095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b17a6-aa2f-4f95-90dc-19c210e40f1e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:26.931227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b17a6-aa2f-4f95-90dc-19c210e40f1e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:26.932489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:26.937424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17caa11-e638-4d63-a44b-55dfdf5b207e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:26.937826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17caa11-e638-4d63-a44b-55dfdf5b207e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:26.937963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17caa11-e638-4d63-a44b-55dfdf5b207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:26.957108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f17caa11-e638-4d63-a44b-55dfdf5b207e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:26.958296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f17caa11-e638-4d63-a44b-55dfdf5b207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:26.966046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:27.124437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b17a6-aa2f-4f95-90dc-19c210e40f1e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:27.138757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b17a6-aa2f-4f95-90dc-19c210e40f1e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:27.144105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:27.144687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:27.144966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:27.145243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:27.153505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:27.155052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573a6ac-66a9-4c45-b616-f4c547c6d467 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:27.165189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 763/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:31.040971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:31.041561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:31.041561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:31.042909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:31.048340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5705c1-a0de-42f0-8aaf-6c35b2402f47 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:31.048757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5705c1-a0de-42f0-8aaf-6c35b2402f47 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:31.048904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5705c1-a0de-42f0-8aaf-6c35b2402f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:31.068909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8c5705c1-a0de-42f0-8aaf-6c35b2402f47 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:31.070426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8c5705c1-a0de-42f0-8aaf-6c35b2402f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:31.086266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:31.373060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:31.374834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:31.452335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a790ce-1906-4db3-9f2a-5ba9d661d8f0 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:31.455922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:31.456516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:31.456795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:31.457093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:31.466035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:31.467492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b26676-4efe-4bd7-a297-969b319695db service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:31.477282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 766/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:33.382595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b9ef3-355d-4800-ab94-3d6ea4be6b80 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:33.383182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b9ef3-355d-4800-ab94-3d6ea4be6b80 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:33.383182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b9ef3-355d-4800-ab94-3d6ea4be6b80 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:33.384492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:33.389703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808a83-157e-4994-8edc-8339811ccc47 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:33.390089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808a83-157e-4994-8edc-8339811ccc47 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:33.390269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808a83-157e-4994-8edc-8339811ccc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:33.409664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-49808a83-157e-4994-8edc-8339811ccc47 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:33.410949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-49808a83-157e-4994-8edc-8339811ccc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:33.418743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:33.711521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b9ef3-355d-4800-ab94-3d6ea4be6b80 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:33.722223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b9ef3-355d-4800-ab94-3d6ea4be6b80 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:33.726632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:33.727246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:33.727526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:33.727799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:33.736751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:33.738179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b5922-ce09-4af7-a093-7f26c11a40f0 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:33.748204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 766/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:37.113508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.114189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.114189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.115560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:37.120850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ead7f-9d5b-4259-b3f7-3c990491d9ae None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.121295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ead7f-9d5b-4259-b3f7-3c990491d9ae None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.121402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ead7f-9d5b-4259-b3f7-3c990491d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.140722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-443ead7f-9d5b-4259-b3f7-3c990491d9ae None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:37.141897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-443ead7f-9d5b-4259-b3f7-3c990491d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.149615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:37.702918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.704596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.771415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:37.787090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f32af2-93e6-4b02-bcaa-1f70d84a8482 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.790727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.791333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.791616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.791892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.800860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.802387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591561e5-1258-46d6-aef2-5311d83e22d5 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.811615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 769/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:39.880995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:39.881594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:39.881594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:39.882920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:35:39.888095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca8913-ba4f-4d7b-b3cf-c44a4ba9cfb4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:39.888546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca8913-ba4f-4d7b-b3cf-c44a4ba9cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:39.888654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca8913-ba4f-4d7b-b3cf-c44a4ba9cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:39.908555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-35ca8913-ba4f-4d7b-b3cf-c44a4ba9cfb4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:39.910509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-35ca8913-ba4f-4d7b-b3cf-c44a4ba9cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:39.918081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:40.470125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:40.472381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:40.485444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8710a9-870d-4768-8546-128f80c127fe None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:40.489767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:40.490293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:40.490582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:40.490865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:40.505915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:40.507396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d438d-8b0e-4df3-a23c-bb1a16e38e2b service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:40.517027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 769/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:49.903859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=11 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:50.512745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:06.754105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213537-9f19-47c1-9d11-5817a00b0c2f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.754783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84ca8-ec97-4c5a-9661-3aacaef99c1e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.755232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213537-9f19-47c1-9d11-5817a00b0c2f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.755331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84ca8-ec97-4c5a-9661-3aacaef99c1e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.755417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213537-9f19-47c1-9d11-5817a00b0c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.755508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd84ca8-ec97-4c5a-9661-3aacaef99c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.779487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-71213537-9f19-47c1-9d11-5817a00b0c2f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.780074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3fd84ca8-ec97-4c5a-9661-3aacaef99c1e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.781101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-71213537-9f19-47c1-9d11-5817a00b0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.781835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3fd84ca8-ec97-4c5a-9661-3aacaef99c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.792538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 771/1540] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:06.793838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.794152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.794432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.801423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 770/1541] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:06.805419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9fbb-40d8-4032-92fc-51ed0be3aa22 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.805706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9fbb-40d8-4032-92fc-51ed0be3aa22 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.805984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9fbb-40d8-4032-92fc-51ed0be3aa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.834564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.835822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.836401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-364f9fbb-40d8-4032-92fc-51ed0be3aa22 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.837677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-364f9fbb-40d8-4032-92fc-51ed0be3aa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.851720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 771/1542] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:06.853048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b222-19d6-4a69-bf9e-01697dcb31da None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.853449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b222-19d6-4a69-bf9e-01697dcb31da None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.853735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b222-19d6-4a69-bf9e-01697dcb31da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.865635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 772/1543] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:06.869749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:06.892729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e259b222-19d6-4a69-bf9e-01697dcb31da None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.894021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e259b222-19d6-4a69-bf9e-01697dcb31da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.896392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941824a6-1a82-40ff-b4e7-a94e75d683f7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:06.908019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 772/1544] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:06.909576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859f14-c877-451e-bc15-6f877054160b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.909867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859f14-c877-451e-bc15-6f877054160b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.910141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02859f14-c877-451e-bc15-6f877054160b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.935397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:06.936032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.936312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.936579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.937239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:06.937542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:06.939832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:06.944276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-02859f14-c877-451e-bc15-6f877054160b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:06.946091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-02859f14-c877-451e-bc15-6f877054160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:06.960059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 773/1545] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:06.964662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02859f14-c877-451e-bc15-6f877054160b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:06.976095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02859f14-c877-451e-bc15-6f877054160b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:06.977159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 773/1546] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:37:06.978018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:06.986662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:06.987290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:06.987785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:06.987913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:06.988498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:06.988804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:06.990436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:06.990436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:37:06.990436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:37:06.991815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c0867-dade-40b8-8d71-1a5fbdd0fbec admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.005078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b97a9f-f0d7-4670-babc-fd24552f8f36 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.017462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 774/1547] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.019575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.019897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.020145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.040210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.041017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.041301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.042973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.043654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.043965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.045590 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.059110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.060334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.065796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 774/1548] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.067086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.072917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 775/1549] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.074662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.092193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc939e59-9921-4988-85bc-00ca199d6222 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.093856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c608-1f14-4d83-ae61-ace3db0b2c82 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.103522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.103742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.104162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.104277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.104448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.104557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.104724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.104836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.105345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.105460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.106366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:07.106366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:37:07.106366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:37:07.107316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.107884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce3f7-0fee-482e-958d-26a1c2cd3ec6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.150276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 776/1550] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.151717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.158726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 775/1551] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.160231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbef8-6073-4b8e-a24b-3fe8d625c0f0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.160517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbef8-6073-4b8e-a24b-3fe8d625c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.160792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbef8-6073-4b8e-a24b-3fe8d625c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.179790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-715fbef8-6073-4b8e-a24b-3fe8d625c0f0 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.180993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-715fbef8-6073-4b8e-a24b-3fe8d625c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.216533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c62e8-56dc-4cc8-9379-7ba070ec25aa admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.242426 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 776/1552] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.243706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310c33-c0d3-4e85-8672-de2af20eb10b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.243991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310c33-c0d3-4e85-8672-de2af20eb10b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.244264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e310c33-c0d3-4e85-8672-de2af20eb10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.263903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3e310c33-c0d3-4e85-8672-de2af20eb10b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.265048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3e310c33-c0d3-4e85-8672-de2af20eb10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.273735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 777/1553] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.275078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4323a3-6ba5-43ea-8499-f80e9c58cdf7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.275376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4323a3-6ba5-43ea-8499-f80e9c58cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.275643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4323a3-6ba5-43ea-8499-f80e9c58cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.276617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.277231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.277511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.277782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.278402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.280460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449bfc-ccbd-465d-ac21-7e8586b176eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.295133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4d4323a3-6ba5-43ea-8499-f80e9c58cdf7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.296372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4d4323a3-6ba5-43ea-8499-f80e9c58cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.305238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 778/1554] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.308435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 777/1555] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.309584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.309873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.310148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.314072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.314362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.314660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.334186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.335448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.338728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.339944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.346952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:07.399594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 778/1556] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.400412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.409386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 779/1557] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.409701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7a4c8-fb8d-4c17-ac34-18b6f02bf2c7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.410219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.414355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.414984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.415250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.415520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.416180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.417149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:07.417149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:37:07.417149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:37:07.418346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.420202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce5375-7576-414d-868f-106af88e9a8a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.425601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.426256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.426537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.426847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.428174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.430136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffbe1c-b3dd-4878-ae77-d7a9a783bb01 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.440832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 779/1558] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.441762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.449176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:07.453832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76a6d-a81a-4599-b7eb-183d37554327 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.455343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 780/1559] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.456766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f84c4-ffe8-4085-8391-86eb28c06750 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.457436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f84c4-ffe8-4085-8391-86eb28c06750 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.458070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f84c4-ffe8-4085-8391-86eb28c06750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.459879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.460476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.460749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.461041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.461682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.461981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.463577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.475992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 780/1560] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.481775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.482497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-741f84c4-ffe8-4085-8391-86eb28c06750 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.483734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-741f84c4-ffe8-4085-8391-86eb28c06750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.487325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaeb14-efbb-492e-9796-c56703182c94 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.492514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 781/1561] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.493497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097245fc-7cef-4cab-a444-3a71cd1a50c5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.493668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.493877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097245fc-7cef-4cab-a444-3a71cd1a50c5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.494053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097245fc-7cef-4cab-a444-3a71cd1a50c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.494396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.494741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.495157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.495793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.497714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b7de0-83be-4aba-9268-c4cf8998fa62 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.515025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-097245fc-7cef-4cab-a444-3a71cd1a50c5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.516167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-097245fc-7cef-4cab-a444-3a71cd1a50c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.521678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 781/1562] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.523044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.523332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.523606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.526045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 782/1563] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.527428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.527714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.528119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.543852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.545306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.547822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:07.548990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:07.554043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 782/1564] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.554854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.560518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 783/1565] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:07.561425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.564321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b6644-1584-4262-abd8-7ac9c11142b3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.569756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.570425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.570805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.571152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.571793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072dfca1-e75c-4856-9802-606c603963d2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.571903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.573441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.576524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.577124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.577408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.577685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.578330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.580147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.586686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 783/1566] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.586686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.594767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 784/1567] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.594767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.595044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dffd44-7dd4-46ce-8786-b995f19e659a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.599136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.599800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.600154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.600430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.601054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.602981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.603266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb178-28b3-4a3c-ab07-b64ce94656dc admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.607844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.608436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.608717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.608992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.609639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.609949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.611556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.624231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 785/1568] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.625303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.636721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ff0a-0349-4625-b7dc-f680042c7525 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.640506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 784/1569] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.641381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.644850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.645474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.645804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.646141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.646854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.648488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.658607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 786/1570] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.659415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.669193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dab519-aa4e-4953-b254-5811c3398471 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.704019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32851f41-6b14-408e-8f8b-7136bc37a983 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.709065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.709684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.709967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.710248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.710924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.711235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.712835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.724682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 785/1571] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.724682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.731070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.731070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.731070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.731070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.731400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.731400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.731400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.741233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7f3a3-19e9-49bc-af20-915e8f940fd0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.749731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 787/1572] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.749731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.749981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.749981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.749981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.749981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.750345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:07.750345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.750345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.760184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f63423-7a5d-4eda-a498-aa3cf907181f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.762824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 786/1573] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:07.763701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.769160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.770074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.770365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.770706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.771405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.773141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.773726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5ab75-d65d-4a31-99fe-db70a8ccf6e1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.778446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.779072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.779356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.779659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.780298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.782024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.783615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 788/1574] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.784473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.784606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:07.794972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183940ab-b437-406e-9e36-a8f6c4abbfd0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.795178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 787/1575] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.796029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.800227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.800826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.801105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.801509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.802811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.804770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.807282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15057-f59a-409a-b0ec-48b0c9887140 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.811877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.812464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.812744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.813018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.813646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.815603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.832707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 789/1576] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.834046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.841609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 788/1577] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.841609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.851667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05928bc2-2208-4f73-b00c-718dec1574fb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.898726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c2956-1d37-4d10-a008-b16d27513a26 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.903703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.904337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.904637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.904899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.905543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.907252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.909469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.910094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.910375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.910736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.911822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.913771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.918837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 790/1578] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:07.919695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.925043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:07.931117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6328-c5d2-40ac-b2a9-a64ed284cc15 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:07.937035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:07.937869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:07.938772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:07.939045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:07.939720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:07.941642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:07.944452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 789/1579] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.945339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:07.969150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 791/1580] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:07.969955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.014335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1afa78-81ff-4072-b0c2-02c2b8522bdc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.035043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6250c45-29b2-403f-81d5-1ce34a5aedff admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.040386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.041041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.041330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.041608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] PATH_INFO: `/v3/projects/67aafb7c76a54d04b7f06d6f63461ee8/users/543ed99b369640e3813dae5cfb96b386/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.057145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67aafb7c76a54d04b7f06d6f63461ee8, user_id=543ed99b369640e3813dae5cfb96b386, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.058174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:08.058174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:37:08.058174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:37:08.060161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.072921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.073519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.073757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 792/1581] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:07 2022] PUT /identity/v3/projects/67aafb7c76a54d04b7f06d6f63461ee8/users/543ed99b369640e3813dae5cfb96b386/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:08.073905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.074072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.074833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.074973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.076737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.101204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:08.109492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 790/1582] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:07 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.111939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.138711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a0ad-035b-4d24-8863-c6fa970125d9 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.173915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2283b5-61ec-41e8-814f-60b457ac0093 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.179290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.179888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.180166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.180445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.181052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.182997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.194340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.195155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.195451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.195719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] PATH_INFO: `/v3/projects/ee9eb7c5b5f94cd78546a3c1cc3d7cc7/users/b490535729c64eaa8e89bf8eb8d56290/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.204452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 791/1583] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.205640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.212141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9eb7c5b5f94cd78546a3c1cc3d7cc7, user_id=b490535729c64eaa8e89bf8eb8d56290, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.214970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.215848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0442a-2386-40f5-8f13-d8aab9436043 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.217295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:08.221295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.221886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.222172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.222464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.223389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.225747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.232126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 793/1584] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:08 2022] PUT /identity/v3/projects/ee9eb7c5b5f94cd78546a3c1cc3d7cc7/users/b490535729c64eaa8e89bf8eb8d56290/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:08.232859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.236615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 792/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.237402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.247610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72485c01-b9ba-4801-a22f-585816650f9e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.298806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54381a-e3ff-4513-8edf-413cad4481e3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.303793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.304448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.304725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.304994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] PATH_INFO: `/v3/projects/98f176b3cafe45d497dee2f1d031a4ec/users/f28236459bb9492eaa35690f121d4e3c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.305526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.306283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.306718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.307094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.307911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.309906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.325940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f176b3cafe45d497dee2f1d031a4ec, user_id=f28236459bb9492eaa35690f121d4e3c, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.328702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaa6f0-d4c6-431d-aecd-1d2c6ec91074 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.332034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 793/1586] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.336438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.343218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 794/1587] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:08 2022] PUT /identity/v3/projects/98f176b3cafe45d497dee2f1d031a4ec/users/f28236459bb9492eaa35690f121d4e3c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:08.345129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.345486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.345830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.346269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78dfb-a538-4c21-999d-25ba97d8663e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.373237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] MFA Rules not processed for user `543ed99b369640e3813dae5cfb96b386`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:08.374527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:08.409443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.410049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.410448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.410758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.411421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.413834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.424789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:08.439855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 794/1588] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.440671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.450912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 795/1589] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:08.451875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.459195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4822b-a0b5-4ebf-a579-cf601efbc37e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.462701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f9264-45cf-4b3b-a05e-418252eee1a3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.466862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.467418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.467617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.467966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.468306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.468404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.468945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.469121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.469429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.470611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.471285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.472659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ea636-2008-4ca8-a556-8ea28ff2bd5f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.483797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 796/1590] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.485411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.485704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.485978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.497502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 795/1591] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/users => generated 285 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.498332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.506819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:08.508052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:08.508777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5e987-2f3f-4e79-bd7e-28a98db23ace admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.513306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.513891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.514174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.514452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.515127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.515695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 797/1592] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:08.519247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.519866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9f6c0-6075-44aa-9ac6-b2a823bf2ac5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.526774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4275162-6087-43c0-ac38-e73c92405c2f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.530633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 796/1593] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.531425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.532024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.532140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.532302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.532389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.532573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.532704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.533310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.534939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.546229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 798/1594] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.547086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.555564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:08.556755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:08.557257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952ade6-19f6-42ac-86f3-2305c337381d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.561890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.562575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.562922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.563199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.563836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.564627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 797/1595] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:08.565415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.565549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.575339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530c86b-b1de-477b-abde-07dee0d12452 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.578023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 799/1596] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.578915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.580405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.581197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.581490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.581763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.582420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.584812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.588707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137fd1b-c68f-4017-8c02-7fab4ab6708d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.593531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.594115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.594399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.594702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.595340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.596919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.609365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 800/1597] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.610180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.614029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 798/1598] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.615778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.618220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:08.620319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c679-b3d5-448c-9d33-e005e1e38aea admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.631799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:08.681527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.681791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77f668-f686-4952-8e62-740a5ad0c339 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.682429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.682743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.683039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.683679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.685267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.685474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.686076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.686359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.686631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.696415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 801/1599] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.697025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.699669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.701239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.706401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fc29-10b1-4d53-8800-b0f5d180ae60 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.711919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.712079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 799/1600] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:08.712550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.712826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.712926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.713098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] PATH_INFO: `/v3/projects/d8697b4591024e9e95a0ed517541293f/users/3c7fa628aaf846be815f159c854f48e2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.722006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992be9d-cf77-46b3-936a-876cee6c6028 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.729558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8697b4591024e9e95a0ed517541293f, user_id=3c7fa628aaf846be815f159c854f48e2, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.729730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.730226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.730512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.730838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.731473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.732879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.733067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.745357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 800/1601] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.746223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.747709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 802/1602] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:08 2022] PUT /identity/v3/projects/d8697b4591024e9e95a0ed517541293f/users/3c7fa628aaf846be815f159c854f48e2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:08.748666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.762834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac2926-60a1-48df-b088-d43f786488ab admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.818692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.818984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.819218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.819522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.820680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c863e38-52bc-421d-a096-8649629b3e42 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.826908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.827428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.827829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.828205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.828966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.829953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.830769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.831607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.842783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 801/1603] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:08.843696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.843913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 803/1604] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.844755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.855182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864e334-a734-469f-b2f4-9298a8637694 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.856828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08001b-59b0-48a2-9d37-b3100f8819ef admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.859943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.860611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.860887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.861171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] PATH_INFO: `/v3/projects/3d204b26857042e397b91be285fa3ed9/users/4dd615d7af7741b08cc44b6c983fae49/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.861945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.862567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.862946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.863528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.864128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.866333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.879457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d204b26857042e397b91be285fa3ed9, user_id=4dd615d7af7741b08cc44b6c983fae49, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.882846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.893645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 804/1605] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:08.894903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.906284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21de2f-6f5b-417f-a397-55f109969107 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.906416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 802/1606] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:08 2022] PUT /identity/v3/projects/3d204b26857042e397b91be285fa3ed9/users/4dd615d7af7741b08cc44b6c983fae49/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:08.910804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:08.922683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:08.976012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.976883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.977236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.977653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.978548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:08.980379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5d605-5e13-4bfe-ae87-b9c8b7d9c9f4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:08.988374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931ce85-4e6d-452f-a70b-f96442b11a58 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:08.993166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 805/1607] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:08.994739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c955-5656-469e-87dc-52f94e74850e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.995206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c955-5656-469e-87dc-52f94e74850e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.995672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c955-5656-469e-87dc-52f94e74850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:08.997193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:08.998071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:08.998355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:08.998704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] PATH_INFO: `/v3/projects/0b8dd47a9d30492eb7163da543d874cb/users/1541e1614d664891b1952379b003ed71/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.004563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:09.015123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8dd47a9d30492eb7163da543d874cb, user_id=1541e1614d664891b1952379b003ed71, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.027933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.035101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c232c955-5656-469e-87dc-52f94e74850e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.037877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c232c955-5656-469e-87dc-52f94e74850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.049904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 803/1608] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:08 2022] PUT /identity/v3/projects/0b8dd47a9d30492eb7163da543d874cb/users/1541e1614d664891b1952379b003ed71/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:09.051489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.115326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 806/1609] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:09.116474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c955-5656-469e-87dc-52f94e74850e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.126687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c955-5656-469e-87dc-52f94e74850e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.132886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.133498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.133786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.134065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.134766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.136374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dceb93-be2c-4305-bc8d-027316117521 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.147288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad3dda-bcca-4d9f-a873-216cc248da48 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.150834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 807/1610] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:09.150834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.150834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.150834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.156076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.157184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.157695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.158047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] PATH_INFO: `/v3/projects/cc41f505368f4a30bdf1822de4ec66b3/users/144c6d061c8c45c3aac4f1c3bb8ed39e/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.171353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.172631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.178561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc41f505368f4a30bdf1822de4ec66b3, user_id=144c6d061c8c45c3aac4f1c3bb8ed39e, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.181619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa77e3-2315-4b6e-98b8-036a9721394c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.185677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 808/1611] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:09.189825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.201172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773975a-08bb-4e5b-a62c-bb819946aedd None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.210697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 804/1612] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:09 2022] PUT /identity/v3/projects/cc41f505368f4a30bdf1822de4ec66b3/users/144c6d061c8c45c3aac4f1c3bb8ed39e/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:09.212548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.213687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.214460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.258105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.259363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.311331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.312106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.312392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.312692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] PATH_INFO: `/v3/projects/8ac23313a6714c58a44e680eb5df0bee/users/69dc8aa4a4004638819fdbebe640d022/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.349905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac23313a6714c58a44e680eb5df0bee, user_id=69dc8aa4a4004638819fdbebe640d022, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.356709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 805/1613] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:09.357781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.358379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.388570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 809/1614] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:09 2022] PUT /identity/v3/projects/8ac23313a6714c58a44e680eb5df0bee/users/69dc8aa4a4004638819fdbebe640d022/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:09.389847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:09.390776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.457109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7aa00-11d3-4c3e-bc6b-63529ad7a4bf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.517936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.518861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.519525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.519525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.534358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7204-374b-4ec0-ad00-564a8ece9edb admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.543112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.543883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.544169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.544532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.545304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.546968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.559458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 810/1615] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:09.560515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.569171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:09.574549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3c51-89da-4c72-91a8-f145aa6a1829 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.580533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.581355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.581538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.581933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.594850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.596293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f922812-a263-4a4c-99ab-cad26dfdca0a service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.597156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.598964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.607434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 811/1616] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:09.608734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.609022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.609300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.609737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 806/1617] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:09.610566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.619099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d75b53-6101-4a6a-8158-32c241218f87 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.622671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.623273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.623703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.623990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.635907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.637312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc1bf0-b114-45c9-8216-9c527f21695b service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.644859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] MFA Rules not processed for user `144c6d061c8c45c3aac4f1c3bb8ed39e`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.647436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.649541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 807/1618] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:09.651021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.651402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.651728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.683070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] MFA Rules not processed for user `69dc8aa4a4004638819fdbebe640d022`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.684397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.724041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 812/1619] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:09.725099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.745741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3c701-eb02-47dd-95fd-d4fb03177d79 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.751381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.752065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.752343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.752644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] PATH_INFO: `/v3/projects/2e81eecde2a44934841e9842c61122a1/users/aa7b5821f9b045d5a42b75f553070100/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.761397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:09.768912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 808/1620] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:09.772207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.775943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e81eecde2a44934841e9842c61122a1, user_id=aa7b5821f9b045d5a42b75f553070100, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.779888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.802751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 813/1621] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:09 2022] PUT /identity/v3/projects/2e81eecde2a44934841e9842c61122a1/users/aa7b5821f9b045d5a42b75f553070100/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:09.804159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:09.833524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe648c12-8e96-464c-9de0-5c4e387e02c6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.881130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8312-f2fa-4dc4-9145-ce3a49f17b04 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:09.887134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.887575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:09.887807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.888034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.888215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.888318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.888498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.888765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.889425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.891421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38091-4495-435f-9c20-6a7251170f9e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.930976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 814/1622] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:09.933609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:09.934274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:09.934978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:09.976047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:09.977661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:09.978379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] MFA Rules not processed for user `aa7b5821f9b045d5a42b75f553070100`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:09.979922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:09.990489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 809/1623] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:09.991557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.002884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aad25c-c5c6-491e-a349-d77b69119cd9 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.030432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:10.080438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.081197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.081550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.081905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.082833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.085390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.086380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 815/1624] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:10.088547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.105417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cd695-ed37-4475-9e77-34782da5d98d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.108124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:10.114307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.115665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.116477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.116823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.117867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.120164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.121048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 810/1625] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:09 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:10.124132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.139225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0a3b8-b081-4b9a-94b1-c1bea55f402b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.147846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.149280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.149852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.150300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.151297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.155248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.163800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 816/1626] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:10 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:10.168798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.186594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 811/1627] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:10.188877 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.207585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e2ab5-492e-4e49-a4a4-f93f3dd019d3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.274322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519a0a-8c19-413d-8423-265668671c51 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.280440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.281125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.281410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.281684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.282398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.284882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.292674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.294031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.294748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.295253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] PATH_INFO: `/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17/users/49278fd84b4341d58db1d2797405fe59/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.315532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de267db0ce5944a7a6bcb3f9caa04f17, user_id=49278fd84b4341d58db1d2797405fe59, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.320606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.328538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 817/1628] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:10 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:10.334066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.346886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cf170-5394-40cc-80ea-df6ad2d9965d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.352184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 812/1629] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:10 2022] PUT /identity/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17/users/49278fd84b4341d58db1d2797405fe59/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:10.353602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.447146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14ad17-957d-4c84-a460-5bd238ac2b5a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.448974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.450436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.451157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.451797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.453719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.459315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.461723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.462089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.462449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.462932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.463903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.466714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.494955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 813/1630] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:10.501252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.515419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 818/1631] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:10.516701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.517231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295a292-5a74-4935-8e59-649438d7889c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.531667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.532354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.532639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.533061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] PATH_INFO: `/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17/users/49278fd84b4341d58db1d2797405fe59/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.540637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de267db0ce5944a7a6bcb3f9caa04f17, user_id=49278fd84b4341d58db1d2797405fe59, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.541357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b41c1-3197-4e3e-b313-40016d7708ca admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.543642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d898cfe-2e7d-41e4-aa5d-ef216c564b6b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.551100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.552616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.553436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.554009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] PATH_INFO: `/v3/projects/f5d800d95ce0432789bdca414210e778/users/4db9c09569584f12b2b627d2a1381c40/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.575797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 814/1632] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:10 2022] PUT /identity/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17/users/49278fd84b4341d58db1d2797405fe59/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:10.582039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.582579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.584045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.595470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d800d95ce0432789bdca414210e778, user_id=4db9c09569584f12b2b627d2a1381c40, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.599516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.628858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 819/1633] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:10 2022] PUT /identity/v3/projects/f5d800d95ce0432789bdca414210e778/users/4db9c09569584f12b2b627d2a1381c40/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:10.633800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.639628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] MFA Rules not processed for user `49278fd84b4341d58db1d2797405fe59`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:10.642511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:10.658593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:10.732896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:10.764793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919ffe7-76f0-4753-bd92-00fdefd8669b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.780987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.782164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.782164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.782164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.784844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.790389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.827039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 820/1634] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:10.833376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.838266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 815/1635] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:10.843192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:10.858868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c829e-a99c-4b26-89cf-6955dd2494f6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:10.872422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:10.873248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.873636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.874018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] PATH_INFO: `/v3/projects/f5d800d95ce0432789bdca414210e778/users/4db9c09569584f12b2b627d2a1381c40/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.877532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d800d95ce0432789bdca414210e778, user_id=4db9c09569584f12b2b627d2a1381c40, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:10.889710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2a772-b245-497c-be1b-8c4f9ee92109 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:10.935542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 821/1636] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:10 2022] PUT /identity/v3/projects/f5d800d95ce0432789bdca414210e778/users/4db9c09569584f12b2b627d2a1381c40/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:10.937206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:10.937530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:10.938690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:10.982591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] MFA Rules not processed for user `4db9c09569584f12b2b627d2a1381c40`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:10.984215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:11.027741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4edb7c-43a5-4200-affc-6de8141ed727 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:11.120579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 822/1637] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:11.135670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:11.172767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:11.177499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:11.177793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:11.178074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:11.275759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef346b-f3b4-4bc0-a9c1-970201dfe6a5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:11.288337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:11.289003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:11.289267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:11.290029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:11.296881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:11.299402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:11.312505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 816/1638] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:11.335666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:11.337161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:11.356823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 823/1639] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:11.438551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:11.451152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736e1d3-ea46-443c-a23c-b688d654d5ca service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:11.472008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:11.473934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:11.475216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:11.475938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:11.538011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:11.557914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:11.560101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb43d8-8b31-4ad2-a0ec-fd811dcc03d9 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:11.581415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 817/1640] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:12.572272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:12.581367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f44bbd-f56d-4654-aad6-bb4879f92fdd service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:12.593427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:12.595151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:12.595151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:12.595151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:12.637684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:12.729769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:12.733379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45522e-93f2-4403-a715-037464485ace service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:12.846578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 824/1641] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:12.847256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cb47-572d-4e2c-9f8b-ea5b211c8d29 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:12.847256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cb47-572d-4e2c-9f8b-ea5b211c8d29 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:12.847256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cb47-572d-4e2c-9f8b-ea5b211c8d29 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:12.847256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 818/1642] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 15:37:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:12.848942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d9fe0-9deb-461c-a3c0-8a4f2cf53a73 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:12.848942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d9fe0-9deb-461c-a3c0-8a4f2cf53a73 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:12.848942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d9fe0-9deb-461c-a3c0-8a4f2cf53a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:12.912517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b93d9fe0-9deb-461c-a3c0-8a4f2cf53a73 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:12.914310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b93d9fe0-9deb-461c-a3c0-8a4f2cf53a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:13.119350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 825/1643] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:13.127179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acc2df-e28e-42a7-a746-524695aafcb1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.127489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acc2df-e28e-42a7-a746-524695aafcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.128443 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acc2df-e28e-42a7-a746-524695aafcb1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.130347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 819/1644] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 15:37:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:13.136197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.136487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.136851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.175461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:13.177283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:13.188541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.189736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.190042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.211678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 826/1645] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:13.234508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:13.256026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fe51f-9d1d-45a1-91d5-6dc0e610ee21 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:13.264259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:13.264858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] MFA Rules not processed for user `3c7fa628aaf846be815f159c854f48e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:13.264987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.265151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.265429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.268674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:13.291809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:13.294002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:13.337113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 827/1646] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:13.508360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:13.508614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 820/1647] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:13.694557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006b13d-607a-4e77-b324-c8ec74cf361d service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:13.704446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:13.705090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.705371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.705806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.706541 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:13.715029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:13.764301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 828/1648] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:13 2022] POST /identity/v3/projects => generated 405 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:13.769861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:13.789236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a3ff-9018-4ee0-a476-d484cad6d07e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:13.909169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:13.910892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:13.911166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:13.911578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:13.913641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:13.918262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:13.966528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 821/1649] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:13 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:13.966938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:13.981065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:13.995540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1f16-1f3b-42d9-b820-fe72b2dfe5e1 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:14.005269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:14.006566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:14.006716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:14.007289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:14.008703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:14.011783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:14.037989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:14.050244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 829/1650] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:14.054329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c64d03-a1af-42e3-b0d4-eff92bfb62ba admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:14.064918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:14.066144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:14.066144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:14.066529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] PATH_INFO: `/v3/projects/dad6f420cca7474782c49511f3e1e25b/users/b76404518d234ddca134bd385ef72e64/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:14.103026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad6f420cca7474782c49511f3e1e25b, user_id=b76404518d234ddca134bd385ef72e64, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:14.107573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:14.188017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:14.194504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 822/1651] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:14 2022] PUT /identity/v3/projects/dad6f420cca7474782c49511f3e1e25b/users/b76404518d234ddca134bd385ef72e64/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:14.199987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac834-7a51-43ff-8eea-0380e2e393ae admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:14.352979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:14.354324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:14.357096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:14.357294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:14.359445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:14.369827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913b579-dba3-4d57-afb6-b28739b1176c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:14.409287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 830/1652] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:14.413819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:14.439978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35940060-e9d3-4fbb-a171-cb6515b1ae51 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:14.465671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:14.466373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:14.466689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:14.473480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] PATH_INFO: `/v3/projects/dad6f420cca7474782c49511f3e1e25b/users/b76404518d234ddca134bd385ef72e64/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:14.478540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad6f420cca7474782c49511f3e1e25b, user_id=b76404518d234ddca134bd385ef72e64, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:14.481341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29d1cf-da2c-457f-8787-cf30f365f436 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:14.531222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 823/1653] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:14 2022] PUT /identity/v3/projects/dad6f420cca7474782c49511f3e1e25b/users/b76404518d234ddca134bd385ef72e64/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:16.054512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c559b1-8304-40b5-bf6f-5a4188cf57ba None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:16.055071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c559b1-8304-40b5-bf6f-5a4188cf57ba None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:16.055071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c559b1-8304-40b5-bf6f-5a4188cf57ba None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:16.056491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 831/1654] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 15:37:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:16.096811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8bb2-8b12-4b9f-b0bc-31638e56c4dc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:16.097288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8bb2-8b12-4b9f-b0bc-31638e56c4dc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:16.097458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8bb2-8b12-4b9f-b0bc-31638e56c4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:16.135648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-965a8bb2-8b12-4b9f-b0bc-31638e56c4dc None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:16.137747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-965a8bb2-8b12-4b9f-b0bc-31638e56c4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:16.202903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:16.341593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 824/1655] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:16.345180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae12a-54c5-456b-9fd6-8d32487553ea None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:16.345962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae12a-54c5-456b-9fd6-8d32487553ea None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:16.348148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ae12a-54c5-456b-9fd6-8d32487553ea None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:16.352925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 832/1656] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 15:37:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:37:16.359995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:16.359995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:16.360293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:16.388844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:16.390383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:16.406874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 825/1657] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:37:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:16.445075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ae12a-54c5-456b-9fd6-8d32487553ea None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:16.612128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ae12a-54c5-456b-9fd6-8d32487553ea None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:16.626264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:16.627023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:16.627355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:16.627636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:16.657090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:16.658751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb1773-1d07-4d70-b7ae-fc89092127ff service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:16.712049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 833/1658] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:17.980058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:18.168390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d52803-e805-4b1d-bbe2-d1ba6a645ac6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:18.181317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:18.183092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:18.183593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:18.184013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] PATH_INFO: `/v3/users/69dc8aa4a4004638819fdbebe640d022` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:18.186988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=69dc8aa4a4004638819fdbebe640d022)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:18.193420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:18.361036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69dc8aa4a4004638819fdbebe640d022'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:18.362166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69dc8aa4a4004638819fdbebe640d022'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:18.388895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69dc8aa4a4004638819fdbebe640d022'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:18.391165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69dc8aa4a4004638819fdbebe640d022'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:18.418891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69dc8aa4a4004638819fdbebe640d022'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:18.511687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 826/1659] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:17 2022] DELETE /identity/v3/users/69dc8aa4a4004638819fdbebe640d022 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:18.513693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:18.514217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:18.514285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:18.559719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:18.573110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:18.868863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 834/1660] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:18.923977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:18.926621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:18.934730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:18.936399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:19.015567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:19.077938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:19.131447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449377-5c65-4da2-8458-bcb2c7197e2b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:19.141892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:19.143885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:19.144295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:19.147523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:19.150795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:19.195477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acadb5d-bb84-4b46-9dcc-61a226df4fcd None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:19.205486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:19.206351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:19.206809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:19.207323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:19.227258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:19.229089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c917b7d-38ff-4e18-9eb7-50b9216996c0 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:19.254026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 835/1661] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:19.256458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbf39c-1487-4f0b-9d51-3f7c18f9951f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:19.256924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbf39c-1487-4f0b-9d51-3f7c18f9951f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:19.257962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbf39c-1487-4f0b-9d51-3f7c18f9951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:19.323961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-47dbf39c-1487-4f0b-9d51-3f7c18f9951f None None] MFA Rules not processed for user `b76404518d234ddca134bd385ef72e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:19.325184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-47dbf39c-1487-4f0b-9d51-3f7c18f9951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:19.336682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:19.338473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f9b19-dacc-4133-81a6-a5b1c9f6a444 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:19.357413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 827/1662] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:19.497441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 836/1663] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:22.781760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e6586-96b9-4f2b-96a9-6ba1004ff267 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:22.782253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e6586-96b9-4f2b-96a9-6ba1004ff267 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:22.782449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e6586-96b9-4f2b-96a9-6ba1004ff267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:22.852951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5f8e6586-96b9-4f2b-96a9-6ba1004ff267 None None] MFA Rules not processed for user `b490535729c64eaa8e89bf8eb8d56290`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:22.855848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5f8e6586-96b9-4f2b-96a9-6ba1004ff267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.010741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 828/1664] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:23.021338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.021338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.021338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.021338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 837/1665] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:23.025842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6af4-52fe-4336-bbde-4608edef949a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.026137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6af4-52fe-4336-bbde-4608edef949a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.026411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de6af4-52fe-4336-bbde-4608edef949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.049606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:23.051464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.109515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-33de6af4-52fe-4336-bbde-4608edef949a None None] MFA Rules not processed for user `b490535729c64eaa8e89bf8eb8d56290`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:23.110898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-33de6af4-52fe-4336-bbde-4608edef949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.111592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60083bde-b103-4a4e-b7a6-270e3af9730b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:23.136842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:23.139933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.140425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.140536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 829/1666] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:23.142051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] PATH_INFO: `/v3/projects/8ac23313a6714c58a44e680eb5df0bee` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.146485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.147005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.148267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.150222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac23313a6714c58a44e680eb5df0bee)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:23.155988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:23.228014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:23.229738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.325080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac23313a6714c58a44e680eb5df0bee', 'request_id': 'req-8dcb0941-76fe-40e2-a158-1d928ccf88ba'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.340823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac23313a6714c58a44e680eb5df0bee', 'request_id': 'req-8dcb0941-76fe-40e2-a158-1d928ccf88ba'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.352256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac23313a6714c58a44e680eb5df0bee', 'request_id': 'req-8dcb0941-76fe-40e2-a158-1d928ccf88ba'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.378633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] The token cache is being invalidate because project 8ac23313a6714c58a44e680eb5df0bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:23.379111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.380569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.385446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 838/1667] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:23 2022] DELETE /identity/v3/projects/8ac23313a6714c58a44e680eb5df0bee => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:23.385446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:23.392403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.439468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 830/1668] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:23.441915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:23.448446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.459686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:23.591632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:23.631647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb0941-76fe-40e2-a158-1d928ccf88ba admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:23.643499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:23.644165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.644990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.645316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] PATH_INFO: `/v3/users/543ed99b369640e3813dae5cfb96b386` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.646470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] RBAC: Authorizing `identity:delete_user(user_id=543ed99b369640e3813dae5cfb96b386)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:23.647909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: WARNING py.warnings [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:23.647909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: warnings.warn(msg) Aug 24 15:37:23.647909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: Aug 24 15:37:23.649151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:23.727028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e0824-eae2-4e95-acd5-ea3b3b2d6317 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:23.875814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '543ed99b369640e3813dae5cfb96b386'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.877191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '543ed99b369640e3813dae5cfb96b386'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.880081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '543ed99b369640e3813dae5cfb96b386'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.885261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:23.887636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:23.887938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:23.888224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:23.894014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:23.900138 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '543ed99b369640e3813dae5cfb96b386'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.945743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '543ed99b369640e3813dae5cfb96b386'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:23.993524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 839/1669] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:23 2022] DELETE /identity/v3/users/543ed99b369640e3813dae5cfb96b386 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:23.994216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:24.006261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.093468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:24.103184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14e459-a1f7-4cd7-9c5f-3188cf8059eb service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:24.142044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 831/1670] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:24.148173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650934b0-76d7-4d19-b781-6d56e52520fa None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.150440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650934b0-76d7-4d19-b781-6d56e52520fa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.151455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650934b0-76d7-4d19-b781-6d56e52520fa None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.159830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 832/1671] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 15:37:24 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:24.169952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22bdac-df67-4f0a-8a1a-5696a63adf4e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.170545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22bdac-df67-4f0a-8a1a-5696a63adf4e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.170979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22bdac-df67-4f0a-8a1a-5696a63adf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.239546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ab22bdac-df67-4f0a-8a1a-5696a63adf4e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:24.241017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ab22bdac-df67-4f0a-8a1a-5696a63adf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.259085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e38622-e2bd-4c87-88e9-09375889610d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:24.270074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 833/1672] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:24.271532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.271822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.272104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.277863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:24.280105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.282725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.283043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] PATH_INFO: `/v3/users/49278fd84b4341d58db1d2797405fe59` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.286955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49278fd84b4341d58db1d2797405fe59)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:24.290306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:24.364692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:24.366082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.457888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49278fd84b4341d58db1d2797405fe59'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:24.458607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49278fd84b4341d58db1d2797405fe59'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:24.459309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49278fd84b4341d58db1d2797405fe59'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:24.464904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:24.493139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49278fd84b4341d58db1d2797405fe59'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:24.511791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:24.524898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49278fd84b4341d58db1d2797405fe59'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:24.548800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 834/1673] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:24.549817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:24.552488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.553683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 840/1674] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:23 2022] DELETE /identity/v3/users/49278fd84b4341d58db1d2797405fe59 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:24.565259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:24.703583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7322cfa-4d55-49b7-8029-fdae1705e4a1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:24.775561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971fe0-7ef9-4d02-9ad7-8087ceccd31f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:24.793609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:24.794830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.794830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.795159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.803264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.843576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:24.845549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:24.879818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 841/1675] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:24.895571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:24.897150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:24.898286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:24.898585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:24.905391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:24.910721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:24.912365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:25.100470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0e268-a7e0-405a-b5fc-65bce1d3f719 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:25.100734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:25.102487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54651c-9391-4309-bd43-6df4d0dee6b1 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:25.130502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:25.131196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:25.131584 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:25.131755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] PATH_INFO: `/v3/users/144c6d061c8c45c3aac4f1c3bb8ed39e` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:25.146367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 835/1676] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:25.147183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=144c6d061c8c45c3aac4f1c3bb8ed39e)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:25.149222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:25.256870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144c6d061c8c45c3aac4f1c3bb8ed39e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:25.257644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144c6d061c8c45c3aac4f1c3bb8ed39e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:25.258684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144c6d061c8c45c3aac4f1c3bb8ed39e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:25.273994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144c6d061c8c45c3aac4f1c3bb8ed39e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:25.308434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144c6d061c8c45c3aac4f1c3bb8ed39e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:25.334767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:25.362015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 842/1677] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:24 2022] DELETE /identity/v3/users/144c6d061c8c45c3aac4f1c3bb8ed39e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:25.367369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a126e-40e4-410a-8d14-d885faed01de None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:25.367557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a126e-40e4-410a-8d14-d885faed01de None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:25.367841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a126e-40e4-410a-8d14-d885faed01de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:25.409499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f76a126e-40e4-410a-8d14-d885faed01de None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:25.412157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f76a126e-40e4-410a-8d14-d885faed01de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:25.528508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:25.584701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 836/1678] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:25.786294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:25.788065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:25.939317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffd90a-6b96-4853-b91c-e320f3db6c79 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:25.948098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:25.949135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:25.950834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:25.951149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:25.978029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:25.982510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da49f40-d913-425d-899a-9ea56e74cd74 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:26.008609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 843/1679] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:26.236884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.237404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.237903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.247256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800d1e-b146-4c7c-8efa-dee380e4d15f None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.247593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800d1e-b146-4c7c-8efa-dee380e4d15f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.248169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800d1e-b146-4c7c-8efa-dee380e4d15f None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.249256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 844/1680] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 15:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:37:26.262864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.263513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.263513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.365436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:26.367130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] MFA Rules not processed for user `f28236459bb9492eaa35690f121d4e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:26.368286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:26.370219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:26.375944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 845/1681] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:26.542413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:26.560219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4719ff6-0757-445f-aa92-5aab39b1a47b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:26.573225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:26.574020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.574437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.574845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.598312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:26.601098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:26.623135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 846/1682] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:26.627825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 837/1683] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:26.646831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:26.648588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:26.829390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5ef14-d8e3-489b-bcba-16978e5192c1 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:26.837632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:26.838730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.839500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.840600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.853110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:26.855185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf483d-043a-49bf-bf38-a7f51db2228f service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:26.872380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 847/1684] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:26.958551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:26.960766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:26.986157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ce84f-812c-4a0d-b632-462523e34c98 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:26.992220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:26.993129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:26.993434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:26.993716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] PATH_INFO: `/v3/projects/67aafb7c76a54d04b7f06d6f63461ee8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:26.996163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67aafb7c76a54d04b7f06d6f63461ee8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:26.997246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:37:26.997246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Aug 24 15:37:26.997246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: Aug 24 15:37:26.998509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:27.000409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:27.000735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:27.001014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:27.002427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 848/1685] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:37:27.117608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67aafb7c76a54d04b7f06d6f63461ee8', 'request_id': 'req-5b827ae9-34ca-4845-bbd8-f1e02e619616'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.125851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67aafb7c76a54d04b7f06d6f63461ee8', 'request_id': 'req-5b827ae9-34ca-4845-bbd8-f1e02e619616'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.146576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67aafb7c76a54d04b7f06d6f63461ee8', 'request_id': 'req-5b827ae9-34ca-4845-bbd8-f1e02e619616'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.166206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] The token cache is being invalidate because project 67aafb7c76a54d04b7f06d6f63461ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:27.166515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.167280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5b827ae9-34ca-4845-bbd8-f1e02e619616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.172130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 838/1686] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:26 2022] DELETE /identity/v3/projects/67aafb7c76a54d04b7f06d6f63461ee8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:27.173422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:27.178147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:27.252087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:27.252600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:27.252674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:27.310411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] MFA Rules not processed for user `5edde502e37144e1b36c7fcfadb6acdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:27.319870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:27.338058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:27.369952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a31e5b-35c2-44d1-b3af-266ce209a78c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:27.389025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:27.389739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:27.390251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:27.390528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] PATH_INFO: `/v3/users/aa7b5821f9b045d5a42b75f553070100` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:27.395754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7b5821f9b045d5a42b75f553070100)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:27.398233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:27.433395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:27.501821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 839/1687] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:27.526070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7b5821f9b045d5a42b75f553070100'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.527670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7b5821f9b045d5a42b75f553070100'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.528513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7b5821f9b045d5a42b75f553070100'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.541395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7b5821f9b045d5a42b75f553070100'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.561415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7b5821f9b045d5a42b75f553070100'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.579227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 849/1688] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:27 2022] DELETE /identity/v3/users/aa7b5821f9b045d5a42b75f553070100 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:27.645476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:27.646989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:27.670444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d8656-2c09-44cc-b64c-66be61e28e91 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:27.778540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:27.779340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:27.779820 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:27.780139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] PATH_INFO: `/v3/users/3c7fa628aaf846be815f159c854f48e2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:27.781923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7fa628aaf846be815f159c854f48e2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:27.785922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:27.795179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:27.797012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:27.949212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf8bff-01da-4713-bc08-952ad8f9099c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:27.951747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7fa628aaf846be815f159c854f48e2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.972993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7fa628aaf846be815f159c854f48e2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:27.980646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:27.982339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:27.985065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:27.985630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:27.997741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:28.015861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7fa628aaf846be815f159c854f48e2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.021285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7fa628aaf846be815f159c854f48e2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.047105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7fa628aaf846be815f159c854f48e2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.052559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:28.056380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:28.106048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 850/1689] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:28.108136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 840/1690] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:27 2022] DELETE /identity/v3/users/3c7fa628aaf846be815f159c854f48e2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:28.158685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:28.163760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:28.190673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f35b7-1063-41c0-af9b-7a6d5f263c09 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:28.350802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:28.352410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:28.446163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:28.454831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:28.455842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:28.456384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:28.457346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] PATH_INFO: `/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:28.459959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de267db0ce5944a7a6bcb3f9caa04f17)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:28.463159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:28.515823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:28.527453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae693ee6-a8b4-435f-8814-8a7b05371a28 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:28.545565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:28.546198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:28.546486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:28.547092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:28.551994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:28.581295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:28.587362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a27dc-6e83-431d-aecc-15c01cad072f service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:28.616027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 841/1691] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:28.647888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de267db0ce5944a7a6bcb3f9caa04f17', 'request_id': 'req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.670735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de267db0ce5944a7a6bcb3f9caa04f17', 'request_id': 'req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.692532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de267db0ce5944a7a6bcb3f9caa04f17', 'request_id': 'req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.717084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] The token cache is being invalidate because project de267db0ce5944a7a6bcb3f9caa04f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:28.717365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.718334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28859f2d-7cd4-4a20-b80f-2c687ef2e29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:28.721993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 851/1692] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:28 2022] DELETE /identity/v3/projects/de267db0ce5944a7a6bcb3f9caa04f17 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:28.727258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:28.727657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:28.727989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:28.803801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:28.811189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:28.997705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615848-4176-4050-8967-baaf41f9a40e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:28.998050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615848-4176-4050-8967-baaf41f9a40e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:28.998267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25615848-4176-4050-8967-baaf41f9a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.001733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 842/1693] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:29.035710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-25615848-4176-4050-8967-baaf41f9a40e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:29.036963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-25615848-4176-4050-8967-baaf41f9a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.066266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.066981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 852/1694] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:29.082846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d988c-fea8-444b-a6bf-857ce85d82a2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.108256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.109152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.109569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.109948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.111023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:29.111436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.119823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.138160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25615848-4176-4050-8967-baaf41f9a40e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.139625 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-25615848-4176-4050-8967-baaf41f9a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.150624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 843/1695] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:29.164188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.169256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25615848-4176-4050-8967-baaf41f9a40e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.174786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.175931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.176226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.176563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] PATH_INFO: `/v3/projects/cc41f505368f4a30bdf1822de4ec66b3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.177746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc41f505368f4a30bdf1822de4ec66b3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.179736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbdbc-f916-415d-97fc-fbd0e92a4291 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.179867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.190058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.191230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.191776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.192249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.193102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.195447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.263263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 844/1696] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:29.284286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.349212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc41f505368f4a30bdf1822de4ec66b3', 'request_id': 'req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.366128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc41f505368f4a30bdf1822de4ec66b3', 'request_id': 'req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.385763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc41f505368f4a30bdf1822de4ec66b3', 'request_id': 'req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.410470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e3d91-8c4f-4a55-a764-0d5314c990fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.414952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] The token cache is being invalidate because project cc41f505368f4a30bdf1822de4ec66b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:29.415871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.416777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ed6153f1-1eb0-4ef5-b443-b0c92ef46376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.418955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 853/1697] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:29 2022] DELETE /identity/v3/projects/cc41f505368f4a30bdf1822de4ec66b3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:29.420574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80da1-7472-45c3-b9e1-2467c376c2be None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.421089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80da1-7472-45c3-b9e1-2467c376c2be None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.421569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80da1-7472-45c3-b9e1-2467c376c2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.504301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b3f80da1-7472-45c3-b9e1-2467c376c2be None None] MFA Rules not processed for user `4dd615d7af7741b08cc44b6c983fae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:29.505626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b3f80da1-7472-45c3-b9e1-2467c376c2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.594468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.595277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.595562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.595840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.596632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.599732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.625800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:29.633335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:29.655391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 845/1698] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/users => generated 294 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:29.660375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.671577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.700332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 854/1699] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:29.700332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.700332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.700332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.700806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0661a-9bad-4c38-88b2-bfdc485cbf49 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.719502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.720526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.722980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.723516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] PATH_INFO: `/v3/projects/2e81eecde2a44934841e9842c61122a1` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.727479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e81eecde2a44934841e9842c61122a1)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.730223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.745410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:29.746699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.777296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 855/1700] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:29.777296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:29.779893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.807260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457101ff-400c-4766-ae5d-36bb8ae0659a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:29.817889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:29.818540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.818887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.819199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.819914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:29.825997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9bf0b-77fb-4752-945e-a4ff7d2e6447 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:29.852293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 856/1701] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:29.855398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e81eecde2a44934841e9842c61122a1', 'request_id': 'req-aca69af0-a475-47fe-a020-77666a9c33b8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.855705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.856055 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.856518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.866949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e81eecde2a44934841e9842c61122a1', 'request_id': 'req-aca69af0-a475-47fe-a020-77666a9c33b8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.876637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e81eecde2a44934841e9842c61122a1', 'request_id': 'req-aca69af0-a475-47fe-a020-77666a9c33b8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.898499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] The token cache is being invalidate because project 2e81eecde2a44934841e9842c61122a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:29.900176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.900954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aca69af0-a475-47fe-a020-77666a9c33b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:29.903934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 846/1702] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:29 2022] DELETE /identity/v3/projects/2e81eecde2a44934841e9842c61122a1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:29.906913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45812e26-a706-4767-8c55-9b2bcac82bb3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:29.907424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45812e26-a706-4767-8c55-9b2bcac82bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:29.907715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45812e26-a706-4767-8c55-9b2bcac82bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:29.930284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] MFA Rules not processed for user `1541e1614d664891b1952379b003ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:29.931723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:29.934064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-45812e26-a706-4767-8c55-9b2bcac82bb3 None None] MFA Rules not processed for user `4dd615d7af7741b08cc44b6c983fae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:29.935370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-45812e26-a706-4767-8c55-9b2bcac82bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.055674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 857/1703] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:30.056944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.058599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.065742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 847/1704] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:30.067005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:30.067498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:30.067750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:30.121579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:30.123503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.162062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c886b4-0c4d-4320-b1f7-488ef81be278 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:30.168145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:30.169363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:30.169723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:30.170119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] PATH_INFO: `/v3/projects/d8697b4591024e9e95a0ed517541293f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:30.171574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8697b4591024e9e95a0ed517541293f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:30.174229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:30.273924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 848/1705] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:30.275290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.281199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.283175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8697b4591024e9e95a0ed517541293f', 'request_id': 'req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.291542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8697b4591024e9e95a0ed517541293f', 'request_id': 'req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.298600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8697b4591024e9e95a0ed517541293f', 'request_id': 'req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.301847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:30.313836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e374-1e7c-4328-9e4e-02482548425b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:30.328404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] The token cache is being invalidate because project d8697b4591024e9e95a0ed517541293f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:30.328850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.329682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.336896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 858/1706] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:30 2022] DELETE /identity/v3/projects/d8697b4591024e9e95a0ed517541293f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:30.337963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.343415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.352017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:30.455566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:30.456279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:30.457115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:30.457396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:30.462353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:30.464066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:30.490883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 849/1707] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:30.491944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.494100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.553305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43bfb5-b526-4e1f-b4b8-80e8a81788b7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:30.563219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:30.564190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:30.564190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:30.566227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] PATH_INFO: `/v3/users/4db9c09569584f12b2b627d2a1381c40` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:30.566413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db9c09569584f12b2b627d2a1381c40)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:30.576401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:30.677774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33ef9f-d4a4-43d0-a4b7-d0035fb89601 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:30.696470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db9c09569584f12b2b627d2a1381c40'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.698730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db9c09569584f12b2b627d2a1381c40'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.700738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:30.701187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db9c09569584f12b2b627d2a1381c40'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.703708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:30.703996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:30.704300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:30.709830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.716797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db9c09569584f12b2b627d2a1381c40'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.754581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db9c09569584f12b2b627d2a1381c40'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:30.784345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 859/1708] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:30 2022] DELETE /identity/v3/users/4db9c09569584f12b2b627d2a1381c40 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:30.786924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.795743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.865293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:30.869156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:30.903785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 850/1709] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:30.904993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:30.913024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:30.941579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf246aa6-704f-48b9-9c66-44baf836882b service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.006129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29469df6-82f1-4f10-85b0-8fa29d1db900 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.023954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.024810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.025510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.025924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.028411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:31.042485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:31.102032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:31.118675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.120575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.121863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.122591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.122923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.123851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.124589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:31.124902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.126527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.159971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 851/1710] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:31.159971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.163388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:31.170915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 860/1711] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:31.173679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.178622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:31.209009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2833e-e089-4f96-bc18-c3fe13bafa22 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.213336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e4d6d-f294-4370-9298-6b59da53b4ae service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.219350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.220230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.220741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.221147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] PATH_INFO: `/v3/projects/cf7d3ab6c1e54f708e112a7a846deebd/users/d409f109e6e247f0ab9d5889f2f22256/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.233024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.233896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.234256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.234524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] PATH_INFO: `/v3/users/b76404518d234ddca134bd385ef72e64` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.238369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76404518d234ddca134bd385ef72e64)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.242970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.282418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7d3ab6c1e54f708e112a7a846deebd, user_id=d409f109e6e247f0ab9d5889f2f22256, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.286495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.324862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 861/1712] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:31 2022] PUT /identity/v3/projects/cf7d3ab6c1e54f708e112a7a846deebd/users/d409f109e6e247f0ab9d5889f2f22256/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:31.331779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.363250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76404518d234ddca134bd385ef72e64'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:31.364737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76404518d234ddca134bd385ef72e64'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:31.380950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76404518d234ddca134bd385ef72e64'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:31.383977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76404518d234ddca134bd385ef72e64'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:31.402739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1a8b93d0-60c6-4109-a1b6-18377a8b340b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76404518d234ddca134bd385ef72e64'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:31.446178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 852/1713] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:31 2022] DELETE /identity/v3/users/b76404518d234ddca134bd385ef72e64 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:31.449698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.451685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.452292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.495785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] MFA Rules not processed for user `d409f109e6e247f0ab9d5889f2f22256`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:31.497533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:31.583141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0428b-0b94-44f7-aae8-d70c1829653a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.593473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 853/1714] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:31.594661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.596317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:31.671203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:31.695457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52bf48-083b-4641-b954-6905d1c008c1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.706784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.708059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.708459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.709018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.710154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.712774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.755885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 854/1715] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:31 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:31.757237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.851472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.855572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.855873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.856156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.856821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.864621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.903944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50116-9035-4290-a5e5-e29cfd76e238 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:31.929173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:31.929890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:31.930180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:31.930916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:31.931579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:31.934003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:31.938742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 862/1716] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:31 2022] POST /identity/v3/projects => generated 365 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:31.953122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:31.964420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:32.003028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 855/1717] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:31 2022] POST /identity/v3/users => generated 299 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:32.007194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.029114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911c652-2e0e-4011-93e2-2f0388de05f9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.038656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:32.156451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaafb2a-78f0-480c-b7c6-95fd4ab43d7a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.175832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.176082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.176484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.176585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.176770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.176865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.177043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.177130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.177767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.179779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.181004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.183729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.225054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 856/1718] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:32.228997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.242252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 863/1719] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:31 2022] POST /identity/v3/users => generated 283 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:32.243557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.249990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5993a5-8871-4e3c-9197-a7766a7ff24c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.264703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc16f-6ca6-4be1-89de-f69e65ec7e8c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.267444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.268303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.269226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.269586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] PATH_INFO: `/v3/projects/66f002821023441098a3b3d65c384bcc/users/1cbf4a1c559f4a20a283a22a8e97e045/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.279888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.280856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.282716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.283096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.283848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.288728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.303574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f002821023441098a3b3d65c384bcc, user_id=1cbf4a1c559f4a20a283a22a8e97e045, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.313254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 864/1720] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:32.314277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.323389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.329844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb543d-84d6-4837-a5d5-0bb1916bed8a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.336333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.338542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.338939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.339442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.343321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.346620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.375517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 857/1721] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:32 2022] PUT /identity/v3/projects/66f002821023441098a3b3d65c384bcc/users/1cbf4a1c559f4a20a283a22a8e97e045/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:32.376630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.390630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 865/1722] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:32.391837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.431724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9145-7d8a-4a8d-90ca-fd7a1153f67f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.501330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:32.562564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1b58-3748-4406-b69f-07894beee69e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.574108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.574900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.575024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.575428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.576062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.580308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e6fd6-985e-4146-9cca-50fff90c4418 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.596627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.600002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.600293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.603372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] PATH_INFO: `/v3/projects/136f2f6e8068434abdd4eeca07614f17/users/5f4d307d85194260a9fb6e4c12686893/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.608190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 858/1723] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:32.611111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d23fa-5a46-4bda-8d3f-7734192a49e0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.611451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d23fa-5a46-4bda-8d3f-7734192a49e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.611728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d23fa-5a46-4bda-8d3f-7734192a49e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.628958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=136f2f6e8068434abdd4eeca07614f17, user_id=5f4d307d85194260a9fb6e4c12686893, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.632685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.676214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 866/1724] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:32 2022] PUT /identity/v3/projects/136f2f6e8068434abdd4eeca07614f17/users/5f4d307d85194260a9fb6e4c12686893/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:32.676214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:32.794073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-797d23fa-5a46-4bda-8d3f-7734192a49e0 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:32.795466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-797d23fa-5a46-4bda-8d3f-7734192a49e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:32.906635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830476f-414c-4c2e-8c75-3c29154f0c9b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:32.927796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:32.928514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.928804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.929083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] PATH_INFO: `/v3/projects/66f002821023441098a3b3d65c384bcc/users/1cbf4a1c559f4a20a283a22a8e97e045/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.933396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f002821023441098a3b3d65c384bcc, user_id=1cbf4a1c559f4a20a283a22a8e97e045, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:32.939189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d9bcf-fb54-431b-aacf-48ec26bbbb14 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:32.976825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 867/1725] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:32 2022] PUT /identity/v3/projects/66f002821023441098a3b3d65c384bcc/users/1cbf4a1c559f4a20a283a22a8e97e045/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:32.980564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.980564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.980564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.986417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 859/1726] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:32.987670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:32.989469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:32.989769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:32.997358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:33.118552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] MFA Rules not processed for user `5f4d307d85194260a9fb6e4c12686893`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:33.119849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.126926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] MFA Rules not processed for user `1cbf4a1c559f4a20a283a22a8e97e045`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:33.133802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.298450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 868/1727] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:33.299691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:33.302614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.384512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 860/1728] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:33.392340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:33.405058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.482712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:33.487118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382b78a-2fca-42cd-a67f-cf989d8cf8ca None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:33.495060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:33.495891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:33.496175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:33.496498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] PATH_INFO: `/v3/projects/f5d800d95ce0432789bdca414210e778` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:33.497868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d800d95ce0432789bdca414210e778)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:33.508287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:33.552262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92399d86-4c1e-4ba1-9466-4d193b453324 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:33.557257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:33.558333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:33.558473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:33.558863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] PATH_INFO: `/v3/projects/dad6f420cca7474782c49511f3e1e25b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:33.560238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad6f420cca7474782c49511f3e1e25b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:33.562327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:33.595519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d800d95ce0432789bdca414210e778', 'request_id': 'req-43886939-c5e7-48be-9bb9-633847ac1ee4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.605997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d800d95ce0432789bdca414210e778', 'request_id': 'req-43886939-c5e7-48be-9bb9-633847ac1ee4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.615493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d800d95ce0432789bdca414210e778', 'request_id': 'req-43886939-c5e7-48be-9bb9-633847ac1ee4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.642097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad6f420cca7474782c49511f3e1e25b', 'request_id': 'req-d0ca7fdb-3147-4503-b1e2-9509303ce855'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.642305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] The token cache is being invalidate because project f5d800d95ce0432789bdca414210e778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:33.642610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.644471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.648303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 869/1729] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:33 2022] DELETE /identity/v3/projects/f5d800d95ce0432789bdca414210e778 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:33.649128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:33.649466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad6f420cca7474782c49511f3e1e25b', 'request_id': 'req-d0ca7fdb-3147-4503-b1e2-9509303ce855'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.654217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.656766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad6f420cca7474782c49511f3e1e25b', 'request_id': 'req-d0ca7fdb-3147-4503-b1e2-9509303ce855'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.678507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] The token cache is being invalidate because project dad6f420cca7474782c49511f3e1e25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:33.678507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.678507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:33.678507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 861/1730] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:33 2022] DELETE /identity/v3/projects/dad6f420cca7474782c49511f3e1e25b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:33.679711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:33.681240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.729973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:33.811887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca7fdb-3147-4503-b1e2-9509303ce855 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:33.825084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:33.825905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:33.826224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:33.826594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:33.827447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:33.832512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:33.887748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 862/1731] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:33 2022] POST /identity/v3/projects => generated 363 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:33.889335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:33.894326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:33.959206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886939-c5e7-48be-9bb9-633847ac1ee4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:33.986218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:33.999839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf449d-e42c-4565-928e-c7372ae7e5c8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:34.006832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:34.007857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.008631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.009568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.012721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:34.158137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:34.159523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.159973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.160414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.162558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:34.195207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:34.197433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad4907-f4aa-458b-848d-3e23637b4ee0 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:34.236019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 863/1732] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:34.237986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464a13-e822-490d-9408-ce06a05db357 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.238539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464a13-e822-490d-9408-ce06a05db357 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.238989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464a13-e822-490d-9408-ce06a05db357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.262724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8d464a13-e822-490d-9408-ce06a05db357 None None] MFA Rules not processed for user `f28236459bb9492eaa35690f121d4e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:34.265433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d464a13-e822-490d-9408-ce06a05db357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:34.303565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:34.305211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131235e-848b-40fa-a970-c03d25fb03bc service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:34.341745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 870/1733] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:37:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:37:34.341745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.341745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.341745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.431769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:34.444072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:34.499650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 864/1734] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:34.522204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2274cd-f11a-49f5-9685-44d55ffacca8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.523648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2274cd-f11a-49f5-9685-44d55ffacca8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.524222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2274cd-f11a-49f5-9685-44d55ffacca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.617749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fa2274cd-f11a-49f5-9685-44d55ffacca8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:34.621587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fa2274cd-f11a-49f5-9685-44d55ffacca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:34.694391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:34.845113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 871/1735] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:34.846174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:34.891423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232c94-06e2-4a31-9b4e-fafd27164424 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:34.899998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 865/1736] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:34.914582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494d15-de4f-4c23-8ada-2c2d4b34e0d4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.915367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:34.915890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:34.916172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.916448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.916806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494d15-de4f-4c23-8ada-2c2d4b34e0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:34.919149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494d15-de4f-4c23-8ada-2c2d4b34e0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:34.922558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:34.924547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:34.993595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-19494d15-de4f-4c23-8ada-2c2d4b34e0d4 None None] MFA Rules not processed for user `4dd615d7af7741b08cc44b6c983fae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:34.998219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-19494d15-de4f-4c23-8ada-2c2d4b34e0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.038947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 872/1737] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:34 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:35.038947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.056070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.112409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e900a2-9bad-4e0e-ab61-731b2208a617 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.129756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.130255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.130540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.130846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] PATH_INFO: `/v3/users/d409f109e6e247f0ab9d5889f2f22256` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.132387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:35.141465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d409f109e6e247f0ab9d5889f2f22256)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:35.154201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:35.180961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 866/1738] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:35.191082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49c40-c2f5-48ed-a97a-0a0e886b67a6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.191509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49c40-c2f5-48ed-a97a-0a0e886b67a6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.191951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49c40-c2f5-48ed-a97a-0a0e886b67a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.235832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6ae49c40-c2f5-48ed-a97a-0a0e886b67a6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:35.237981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6ae49c40-c2f5-48ed-a97a-0a0e886b67a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.274719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 867/1739] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:35.285523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.289186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.292208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.353858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd409f109e6e247f0ab9d5889f2f22256'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:35.356015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd409f109e6e247f0ab9d5889f2f22256'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:35.360288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd409f109e6e247f0ab9d5889f2f22256'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:35.382700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd409f109e6e247f0ab9d5889f2f22256'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:35.410017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:35.411671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.414966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd409f109e6e247f0ab9d5889f2f22256'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:35.450188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 868/1740] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:35.450887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.460371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.466680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 873/1741] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:35 2022] DELETE /identity/v3/users/d409f109e6e247f0ab9d5889f2f22256 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:35.467930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.471008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.518605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb77ec1-c2a5-400c-9a63-fe519ca30781 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.707235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0ed9-ccdc-4029-a3e8-b0bef9dd4f8b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.726035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.726679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.726984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.727237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.727898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:35.728198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:35.729814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:35.777061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.777821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.777943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.778338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.778885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:35.780669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b512ef0-7a43-414d-8d7f-2b88543324f5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:35.781447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 874/1742] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:35.782543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.812993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae5ce9-8a66-48bf-b7a1-c394ca8816b8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.819356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 869/1743] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:35.821515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.821794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.822064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.839491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.840118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.840393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.840668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.841329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:35.841632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:35.843279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:35.885054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 875/1744] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:35.885829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.900617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:35.901878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:35.929388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c835370-ffb4-4de0-af8e-9cfc3ae90fa2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.931204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 870/1745] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:35.936986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:35.951420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.952019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.952263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.952607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:35.953166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:35.955111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:35.972046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45583327-b708-42f5-a6e5-f35aa5510491 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:35.986672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:35.990142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:35.990513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:35.990714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] PATH_INFO: `/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a/users/945d28a798a645ceaa4149b6c838f55a/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.053393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9df5ea44fb4ddcbc74927dfb46f67a, user_id=945d28a798a645ceaa4149b6c838f55a, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.054361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:36.056636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.076883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 871/1746] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:35 2022] PUT /identity/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a/users/945d28a798a645ceaa4149b6c838f55a/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:36.078268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.079383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 876/1747] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:35 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:36.080474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.084208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:36.236517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae753-cc74-4d25-814c-8db807409014 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.247771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.250509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.250828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.251108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.286059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82e4f-e84b-4a62-b571-4b819b1b7160 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.390224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:36.403799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.406897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.407738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.408684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.410064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.410762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.411925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.414576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.436800 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 877/1748] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:36.437783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.456097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 872/1749] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:36 2022] POST /identity/v3/projects => generated 371 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:36.457279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.458547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88eb-9776-484c-8da0-541710849549 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.578278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce71a44-8fa2-4049-9f44-844c1d90332f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.586419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.602349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.603032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.606289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.606679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.610528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.615094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.615360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.616093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.616394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.620956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.631552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.657573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 878/1750] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:36.661136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.696435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cc06-9120-48f9-ac0c-97183e3d4c9c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.707066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.707982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.708771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.709162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.710259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.724797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddbcfb-c0ed-4026-a2ac-d10381c8291f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.728857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 873/1751] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:36 2022] POST /identity/v3/users => generated 284 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:36.733586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.774169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018b03b-f87c-4894-b0ad-7541ae884984 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:36.779325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 879/1752] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:36 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:36.781732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.782328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.782836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.790146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:36.791829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:36.792117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:36.793703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] PATH_INFO: `/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a/users/945d28a798a645ceaa4149b6c838f55a/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:36.798865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9df5ea44fb4ddcbc74927dfb46f67a, user_id=945d28a798a645ceaa4149b6c838f55a, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:36.806494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:36.848762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] MFA Rules not processed for user `1541e1614d664891b1952379b003ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:36.849930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:36.856463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 874/1753] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:36 2022] PUT /identity/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a/users/945d28a798a645ceaa4149b6c838f55a/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:36.857570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:36.959687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:37.088375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9c631-dacd-4200-b979-fa8072dc08af admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.104313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.104313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.104313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.104729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.105251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.105816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 880/1754] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:37.106915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4aac77-a786-43f4-ac26-657d7fc18c90 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.109047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.133331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 875/1755] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:37.134589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe03c-13b6-467d-91e5-73649e1c5aa8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.145642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe03c-13b6-467d-91e5-73649e1c5aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.145642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe03c-13b6-467d-91e5-73649e1c5aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.156916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5299ba7-830f-4171-953d-5226e91a8984 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.173314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.173681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.173966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.174286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.175134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.176623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.231431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 881/1756] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:37.232409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.258293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fedfe03c-13b6-467d-91e5-73649e1c5aa8 None None] MFA Rules not processed for user `945d28a798a645ceaa4149b6c838f55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:37.259517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fedfe03c-13b6-467d-91e5-73649e1c5aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:37.265229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610156-a116-4ef1-94ea-22c7e81f840d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.276364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.276988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.277507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.277711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.278359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.288124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.316205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 882/1757] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:37.316538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.340933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883377ac-6100-4867-9e1d-4dfe0f01eec7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.345345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:37.361759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.361999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.362151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.362432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.363212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.364827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.406238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 883/1758] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:37.406238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.431516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f538d-abf2-478c-87ff-e221746414f5 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.445512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 876/1759] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:37.445512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.445512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.445512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.447344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.448287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.448384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.448589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] PATH_INFO: `/v3/projects/4815cb1110cb41c193f0f622a6c02f46/users/305dde7a41194704865342335d92e970/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.483480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4815cb1110cb41c193f0f622a6c02f46, user_id=305dde7a41194704865342335d92e970, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.490497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.494082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:37.499303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:37.524694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 884/1760] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:37 2022] PUT /identity/v3/projects/4815cb1110cb41c193f0f622a6c02f46/users/305dde7a41194704865342335d92e970/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:37.535606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.719379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 877/1761] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 24 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:37.721283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:37.725010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:37.725540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f4d54-e1b1-4431-82d2-be98bbc43c3a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.731346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:37.732103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.732324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.732579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] PATH_INFO: `/v3/projects/856e161dc4cf435ebf1057a2881c0716/users/243dbcc38c474ffc952f0ca74e0a8428/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.779859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=856e161dc4cf435ebf1057a2881c0716, user_id=243dbcc38c474ffc952f0ca74e0a8428, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:37.782568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d7377-78b3-40b6-80f1-150fc76fd6a8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:37.835394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 885/1762] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:37 2022] PUT /identity/v3/projects/856e161dc4cf435ebf1057a2881c0716/users/243dbcc38c474ffc952f0ca74e0a8428/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:37.836679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:37.836967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:37.837239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:37.923015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f1132-afc6-4bbf-a52a-d353049bc5af None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:37.938483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] MFA Rules not processed for user `305dde7a41194704865342335d92e970`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:37.939913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.083964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:38.150425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:38.151626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.152270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.152868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.188427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 886/1763] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:38.189371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:38.198164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.288670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:38.334095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:38.336020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4fcbf-4f58-4222-9633-d6c8b746d94d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:38.336151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:38.341545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:38.342565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.343070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.343354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] PATH_INFO: `/v3/projects/cf7d3ab6c1e54f708e112a7a846deebd` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.344736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7d3ab6c1e54f708e112a7a846deebd)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:38.347202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:38.366622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 878/1764] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:38.368249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:38.468728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7d3ab6c1e54f708e112a7a846deebd', 'request_id': 'req-d4f20e45-2403-483e-96db-44473f0eb784'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:38.475813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7d3ab6c1e54f708e112a7a846deebd', 'request_id': 'req-d4f20e45-2403-483e-96db-44473f0eb784'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:38.483479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7d3ab6c1e54f708e112a7a846deebd', 'request_id': 'req-d4f20e45-2403-483e-96db-44473f0eb784'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:38.511402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] The token cache is being invalidate because project cf7d3ab6c1e54f708e112a7a846deebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:38.533365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3bcdb-14ee-4505-a8b0-fa89c6db640c service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:38.541781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:38.545785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d4f20e45-2403-483e-96db-44473f0eb784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:38.553902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 887/1765] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:38 2022] DELETE /identity/v3/projects/cf7d3ab6c1e54f708e112a7a846deebd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:38.578876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7743-4668-4374-9351-745fed23b1d4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.579179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7743-4668-4374-9351-745fed23b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.580285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7743-4668-4374-9351-745fed23b1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.640012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8bda7743-4668-4374-9351-745fed23b1d4 None None] MFA Rules not processed for user `243dbcc38c474ffc952f0ca74e0a8428`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:38.641445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8bda7743-4668-4374-9351-745fed23b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.677141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:38.677875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.678168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.678452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.680398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.717191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:38.718740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:38.734478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 879/1766] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:38.735499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:38.736915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.814395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 888/1767] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:38.817678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.817957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.818689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.820432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 889/1768] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 15:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:38.821997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:38.823935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:38.931937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:38.958456 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41af06e-66d7-4f0e-bb8b-b000463a3671 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:38.965143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:38.966038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:38.966601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:38.967653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:38.968435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:38.970862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:39.021472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 880/1769] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:38 2022] POST /identity/v3/projects => generated 367 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:39.022492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:39.062071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.160538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:39.161293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2974f-58f2-4932-b47e-6dfba20948af None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:39.341063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbcb04-0a6d-4d73-8e49-8d6b7825371d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:39.348410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:39.350053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.351093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.351548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.352666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:39.356437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9e9d4-9f8d-4eab-beb1-a0e1fe122400 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:39.373404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:39.374213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.374601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.375007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] PATH_INFO: `/v3/users/1cbf4a1c559f4a20a283a22a8e97e045` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.376798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cbf4a1c559f4a20a283a22a8e97e045)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:39.380852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:39.454740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 881/1770] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:39 2022] POST /identity/v3/projects => generated 371 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:39.460562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.463768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.465340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.520418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:39.521827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.549015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cbf4a1c559f4a20a283a22a8e97e045'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:39.551261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cbf4a1c559f4a20a283a22a8e97e045'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:39.552912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cbf4a1c559f4a20a283a22a8e97e045'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:39.569860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cbf4a1c559f4a20a283a22a8e97e045'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:39.597370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cbf4a1c559f4a20a283a22a8e97e045'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:39.638399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 890/1771] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:38 2022] DELETE /identity/v3/users/1cbf4a1c559f4a20a283a22a8e97e045 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:39.645145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:39.648036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.668426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 882/1772] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:37:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:39.669332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:39.673271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.768083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:39.828832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640fe5-1a9f-4822-ae9a-e6775928bb3d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:39.841761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:39.843863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.844414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.844844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.845796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:39.851406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:39.854343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8c2a-0626-4a33-af2d-beb4a7f26c54 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:39.859725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:39.861714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.862200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.863190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.863971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:39.866459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:39.900394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 891/1773] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:39 2022] POST /identity/v3/users => generated 290 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:39.901917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:39.906888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.925992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 883/1774] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:39 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:39.929215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:39.935009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:39.936274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839bc5-345d-4e52-b393-cb484ba1506f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:39.949084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:39.949915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:39.950399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:39.951199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] PATH_INFO: `/v3/users/5f4d307d85194260a9fb6e4c12686893` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:39.952559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4d307d85194260a9fb6e4c12686893)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:39.982829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.059053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8da5-3043-4233-b58c-577a3385cd8b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.063912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.064535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.064819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.065281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.071765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:40.074671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.076147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.091133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 884/1775] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:40.093103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.134233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a825e-3863-4e9c-a589-f9083a96733e service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.142403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.145515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.146106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.147220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.148886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.153107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.172504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 885/1776] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:40.173573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.200079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc776d0a-7ed4-4599-a714-e7b9cc94ba0f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.211050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.212178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.212666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.213224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.214121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.216368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.216525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4d307d85194260a9fb6e4c12686893'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:40.217209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4d307d85194260a9fb6e4c12686893'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:40.218367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4d307d85194260a9fb6e4c12686893'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:40.241036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4d307d85194260a9fb6e4c12686893'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:40.251999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 886/1777] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:40.253835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.269694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:40.302704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4d307d85194260a9fb6e4c12686893'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:40.311660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee6de0-945d-4ff9-a463-e7888a671ef3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.350251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.350939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.351226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.351509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] PATH_INFO: `/v3/projects/aa8d557059fa40b79f17a6aeee1db79d/users/d4fd6a57fbad449a8d309bba9d387a6f/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.378511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 892/1778] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:39 2022] DELETE /identity/v3/users/5f4d307d85194260a9fb6e4c12686893 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:40.379727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.387620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:40.437205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8d557059fa40b79f17a6aeee1db79d, user_id=d4fd6a57fbad449a8d309bba9d387a6f, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.461837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d416b25-7974-4e23-9563-186b5c057661 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.518008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 887/1779] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:40 2022] PUT /identity/v3/projects/aa8d557059fa40b79f17a6aeee1db79d/users/d4fd6a57fbad449a8d309bba9d387a6f/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:40.524386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.524691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.525028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.638601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:40.647577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:40.670362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1b271-8a45-4648-b5bf-89a88f4705ba admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.770303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 888/1780] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:40.771932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.788332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4ae56-cbd4-4782-8a12-2f9fd213937b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.797412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.798506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.799358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.800673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.802040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.804608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.826914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 889/1781] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:40.828705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.849557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a92a6-7dbd-4f07-9d79-2bdb89274061 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:40.867388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.868975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.869538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.870114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] PATH_INFO: `/v3/projects/aa8d557059fa40b79f17a6aeee1db79d/users/d4fd6a57fbad449a8d309bba9d387a6f/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.874837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8d557059fa40b79f17a6aeee1db79d, user_id=d4fd6a57fbad449a8d309bba9d387a6f, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.880551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:40.890294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:40.917484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 890/1782] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:40 2022] PUT /identity/v3/projects/aa8d557059fa40b79f17a6aeee1db79d/users/d4fd6a57fbad449a8d309bba9d387a6f/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:40.930116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:40.933113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:40.963279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:40.965587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:40.965920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:40.967476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] PATH_INFO: `/v3/projects/b2c590404cd24993a3e7a7a22e36ab67/users/cce347a70d5a4181906ce2a5d6f12b91/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:40.991089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c590404cd24993a3e7a7a22e36ab67, user_id=cce347a70d5a4181906ce2a5d6f12b91, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:40.994048 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc0344-135e-49b3-86a1-f2f305858768 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:41.036931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 893/1783] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:40 2022] PUT /identity/v3/projects/b2c590404cd24993a3e7a7a22e36ab67/users/cce347a70d5a4181906ce2a5d6f12b91/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:41.041542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.041900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.044189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.113548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] MFA Rules not processed for user `d4fd6a57fbad449a8d309bba9d387a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:41.117032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:41.117947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d5375-e56e-4626-b963-0b855a029099 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:41.262268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:41.310503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:41.318864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.318864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.318864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.377425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 894/1784] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:41.390836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:41.588582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:41.593261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:41.629452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 891/1785] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:41.635022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:41.638459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:41.681302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a65cf-90a2-4130-ac12-af020eb9fc97 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:41.696957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:41.699324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.699663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.700212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.700936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:41.707298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:41.745229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 895/1786] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:41.747310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:41.780093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec657f-601d-4e0d-8223-3fe516d4c62c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:41.812166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:41.814762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.821821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3083e-ed6d-4701-a9a2-7872532eed90 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:41.822913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.823213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] PATH_INFO: `/v3/projects/b2c590404cd24993a3e7a7a22e36ab67/users/cce347a70d5a4181906ce2a5d6f12b91/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.831812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c590404cd24993a3e7a7a22e36ab67, user_id=cce347a70d5a4181906ce2a5d6f12b91, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:41.839498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c7c70-3c6d-42af-a52e-55cd83c18b93 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:41.842763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:41.843479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.843766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.844050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.856590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:41.858142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:41.872349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 896/1787] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:41 2022] PUT /identity/v3/projects/b2c590404cd24993a3e7a7a22e36ab67/users/cce347a70d5a4181906ce2a5d6f12b91/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:41.882780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 892/1788] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:41.890963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:41.892170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:41.892684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:41.939619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] MFA Rules not processed for user `cce347a70d5a4181906ce2a5d6f12b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:41.943018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:42.007363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:42.079499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:42.094489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98877351-b9d8-419d-b957-a2e41d531b02 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:42.155000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 897/1789] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:42.164737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:42.174114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:42.215199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:42.215893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:42.216937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:42.217229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:42.219228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:42.354533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:42.377819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9147fd1-22fc-4a12-aaae-44d211861c88 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:42.385274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:42.386069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:42.386509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:42.386962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:42.398301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:42.400031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2804d87-d4cd-40c4-a7a3-8b2fed105cf9 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:42.422621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:42.435267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8aaeb4-d3cc-4bc1-b7fc-021f1a6f2b6d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:42.442024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 898/1790] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:42.484502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 893/1791] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:42.850705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8af6f-84f0-4ea8-8e77-bb77aafcab01 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:42.850999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8af6f-84f0-4ea8-8e77-bb77aafcab01 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:42.851275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8af6f-84f0-4ea8-8e77-bb77aafcab01 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:42.853621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 899/1792] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:42.936583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:42.936926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:42.937482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:42.984884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:42.989441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:43.259846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 894/1793] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:37:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:43.557949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8af6f-84f0-4ea8-8e77-bb77aafcab01 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:43.710308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:43.710529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8af6f-84f0-4ea8-8e77-bb77aafcab01 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:43.711473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:43.723169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:43.724143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:43.724392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:43.724661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] PATH_INFO: `/v3/users/243dbcc38c474ffc952f0ca74e0a8428` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:43.727616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=243dbcc38c474ffc952f0ca74e0a8428)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:43.731534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:43.738084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d52802-b574-4d69-9f1d-fac996a0a8a6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:43.750083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:43.751253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:43.751253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:43.751253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] PATH_INFO: `/v3/projects/66f002821023441098a3b3d65c384bcc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:43.756964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f002821023441098a3b3d65c384bcc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:43.761322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:43.859581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243dbcc38c474ffc952f0ca74e0a8428'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:43.863756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243dbcc38c474ffc952f0ca74e0a8428'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:43.864855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243dbcc38c474ffc952f0ca74e0a8428'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:43.897946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243dbcc38c474ffc952f0ca74e0a8428'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:43.928113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:43.937325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab9aa170-f4c8-4718-b528-7674e093c783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243dbcc38c474ffc952f0ca74e0a8428'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:43.989134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 900/1794] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:43 2022] DELETE /identity/v3/users/243dbcc38c474ffc952f0ca74e0a8428 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:43.993777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:43.997334 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:43.997772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:44.032349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f002821023441098a3b3d65c384bcc', 'request_id': 'req-6cdabd3f-de18-42b1-a17a-8344f74b233f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.039240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f002821023441098a3b3d65c384bcc', 'request_id': 'req-6cdabd3f-de18-42b1-a17a-8344f74b233f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.044922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:44.053124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.066931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f002821023441098a3b3d65c384bcc', 'request_id': 'req-6cdabd3f-de18-42b1-a17a-8344f74b233f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.102932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] The token cache is being invalidate because project 66f002821023441098a3b3d65c384bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:44.103408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.104423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6cdabd3f-de18-42b1-a17a-8344f74b233f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.108360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 895/1795] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:43 2022] DELETE /identity/v3/projects/66f002821023441098a3b3d65c384bcc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:44.115786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:44.116072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:44.116346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:44.162934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:44.164188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.221825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 901/1796] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:44.244385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:44.247714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.328505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 896/1797] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:44.329777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:44.336725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.360626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:44.377312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bd831-d5d5-42a6-af84-213eb871f46a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:44.391152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:44.391988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:44.392332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:44.393446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] PATH_INFO: `/v3/projects/136f2f6e8068434abdd4eeca07614f17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:44.395343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136f2f6e8068434abdd4eeca07614f17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:44.397498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:44.492434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3181a1-dbd0-42f6-8afa-24c50884cd1d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:44.524771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:44.525484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:44.529239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:44.529531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:44.575123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:44.576579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be390a-227d-4c16-9f80-4dd61736cf13 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:44.613270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136f2f6e8068434abdd4eeca07614f17', 'request_id': 'req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.624807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 902/1798] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:44.632099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:44.633663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:44.634254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:44.639667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136f2f6e8068434abdd4eeca07614f17', 'request_id': 'req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.666052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136f2f6e8068434abdd4eeca07614f17', 'request_id': 'req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.726930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] The token cache is being invalidate because project 136f2f6e8068434abdd4eeca07614f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:44.727231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.729402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:44.736067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:44.739202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 897/1799] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:44 2022] DELETE /identity/v3/projects/136f2f6e8068434abdd4eeca07614f17 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:44.740620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.741186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:44.743805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.933955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 903/1800] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:44.935251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:44.938229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:44.984666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6740-4924-41c4-a9aa-5e727cbab0ba None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:44.995847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255b117-a0ba-4b8c-b87e-fc248a4cdc36 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.004360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.004360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.004360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.004360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] PATH_INFO: `/v3/users/945d28a798a645ceaa4149b6c838f55a` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.004990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945d28a798a645ceaa4149b6c838f55a)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.006742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.006742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.007042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.007172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.007172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.008739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:45.027208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.029736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.040803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:45.050291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 898/1801] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:45.051316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.089700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f30caf-006e-4e6b-ad03-b38f37eb1431 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.103725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.106677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.106959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.113881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.116000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:45.116823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.120484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.130305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945d28a798a645ceaa4149b6c838f55a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:45.131346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945d28a798a645ceaa4149b6c838f55a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:45.133198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945d28a798a645ceaa4149b6c838f55a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:45.159419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945d28a798a645ceaa4149b6c838f55a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:45.171426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 899/1802] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:45.172379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.180336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:45.203599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945d28a798a645ceaa4149b6c838f55a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:45.222707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f1c23-6a54-4e1d-89e3-3a3089e98148 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.235729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.236355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.236678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.237508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.238341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.240324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.242283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 904/1803] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:44 2022] DELETE /identity/v3/users/945d28a798a645ceaa4149b6c838f55a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:45.304461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 900/1804] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:45 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:45.308920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.519841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e38fca-701a-4eb5-a34f-a7571f5c7022 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.733213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.734574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.735016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.735456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.736249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.738706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.787102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 905/1805] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:45 2022] POST /identity/v3/users => generated 297 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:45.790948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.813974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133657-6158-4355-acf9-e07a41c9ae38 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.832490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.836636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.837002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.837291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.840047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.848476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e3014-7fe0-4dae-b4e4-7023e8f0e31f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.889635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 901/1806] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:45.890812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.911858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece74f9d-13bd-44b6-8a3e-e2122295626c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:45.921289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:45.922048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993daa11-3ee4-4122-8d4f-d1d11781f0dd None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.922332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993daa11-3ee4-4122-8d4f-d1d11781f0dd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.922681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993daa11-3ee4-4122-8d4f-d1d11781f0dd None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.922888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:45.923354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:45.923856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:45.925540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:45.927962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0416d92-5c81-4c56-a975-ce1f54580eb8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:45.930469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 902/1807] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:37:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:37:45.949642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 906/1808] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:45.950927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993daa11-3ee4-4122-8d4f-d1d11781f0dd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:45.979772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993daa11-3ee4-4122-8d4f-d1d11781f0dd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:46.011085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:46.013222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.017094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.017540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] PATH_INFO: `/v3/projects/4498b9ca612d47f7b8f3d8fc34028640/users/93618779e43849ca95463e4d8d033655/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.019552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.019993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.020184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.068738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:46.073891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:46.080969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4498b9ca612d47f7b8f3d8fc34028640, user_id=93618779e43849ca95463e4d8d033655, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:46.088208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bd4c-70e6-4847-8090-372f4ed460e5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:46.133704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 903/1809] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:45 2022] PUT /identity/v3/projects/4498b9ca612d47f7b8f3d8fc34028640/users/93618779e43849ca95463e4d8d033655/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:46.148162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.148450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.148734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.152142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:46.158904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:46.220824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] MFA Rules not processed for user `93618779e43849ca95463e4d8d033655`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:46.223993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:46.358195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 904/1810] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:46.361052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 907/1811] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:37:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:46.362487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:46.386577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:46.388519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:46.432536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec140-5b8c-4379-b2e5-a63b25100045 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:46.616042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:46.619399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.619859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.620575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.647206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:46.648860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693101c-3402-458a-9bbb-ccb55fb7b936 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:46.651280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:46.661029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:46.662080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.662547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.665237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.665237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:46.667512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:46.684629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 908/1812] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:46.771031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:46.774423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 905/1813] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:46 2022] POST /identity/v3/projects => generated 393 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:46.783710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb1b72-15ac-44f8-8bca-40a43e2b5206 service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:46.862122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:46.863037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.863281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.863557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.864195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:46.866176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:46.902627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 909/1814] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:46 2022] POST /identity/v3/users => generated 303 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:46.903228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:46.922809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e68a72-4b95-4701-861f-9e03d74ed4a8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:46.938131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:46.939418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:46.940906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:46.942398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:46.945706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:46.952475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:47.018671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 906/1815] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:47.022243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:47.051096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c45d6b-dc9c-480a-b17e-7f1f418558a3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:47.072760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:47.073448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:47.073781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:47.074109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] PATH_INFO: `/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a/users/0c05a01601834f729e7f202261030927/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:47.128294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee6e4e9af8b48f582e3baa9dccb820a, user_id=0c05a01601834f729e7f202261030927, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:47.131002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:47.181252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 910/1816] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:47 2022] PUT /identity/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a/users/0c05a01601834f729e7f202261030927/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:47.185996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:47.213976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb2982-4d10-43f5-86a8-56b0ce4aad95 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:47.382677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:47.383730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:47.384169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:47.384637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:47.385513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:47.387336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4062f-985a-48fa-9ce2-d5709e512090 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:47.405932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 907/1817] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:47.410702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:47.427712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd84469-5157-43ab-a385-42dc93aa7a0b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:47.454241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:47.455026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:47.455924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:47.456348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] PATH_INFO: `/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a/users/0c05a01601834f729e7f202261030927/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:47.466824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee6e4e9af8b48f582e3baa9dccb820a, user_id=0c05a01601834f729e7f202261030927, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:47.472183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:47.474891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:47.495041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 911/1818] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:47 2022] PUT /identity/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a/users/0c05a01601834f729e7f202261030927/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:47.507435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:47.507703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:47.507990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:47.519072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:47.572201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] MFA Rules not processed for user `0c05a01601834f729e7f202261030927`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:47.579357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:47.734050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 908/1819] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:47.747010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:47.749222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:47.794783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:47.796472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:47.940421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca900c-828b-44d8-a2a5-a81eb8189260 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:47.957337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:47.959964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:47.960456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:47.960895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:47.995360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8251e-ffdc-49f6-911c-85dda400d4cf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:47.997108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:47.998761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:48.010567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:48.011493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.013871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.014017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] PATH_INFO: `/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.015475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9df5ea44fb4ddcbc74927dfb46f67a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:48.022415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:48.036355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 912/1820] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:47 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:48.182686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:48.256887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e88c4-b0e2-4c09-8254-a19ef6032063 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:48.323101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9df5ea44fb4ddcbc74927dfb46f67a', 'request_id': 'req-54c15f98-6a23-4661-87bc-ff27273e6cd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:48.350251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9df5ea44fb4ddcbc74927dfb46f67a', 'request_id': 'req-54c15f98-6a23-4661-87bc-ff27273e6cd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:48.387402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9df5ea44fb4ddcbc74927dfb46f67a', 'request_id': 'req-54c15f98-6a23-4661-87bc-ff27273e6cd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:48.425175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] The token cache is being invalidate because project 8a9df5ea44fb4ddcbc74927dfb46f67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:48.425175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:48.429392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54c15f98-6a23-4661-87bc-ff27273e6cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:48.432753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 909/1821] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:47 2022] DELETE /identity/v3/projects/8a9df5ea44fb4ddcbc74927dfb46f67a => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:48.441172 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df5fc4-dc03-40e0-97ad-8dee3a8b72b6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.441463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df5fc4-dc03-40e0-97ad-8dee3a8b72b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.441750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df5fc4-dc03-40e0-97ad-8dee3a8b72b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.483582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:48.486218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.487272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.487575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.506203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:48.550500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a0df5fc4-dc03-40e0-97ad-8dee3a8b72b6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:48.555493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a0df5fc4-dc03-40e0-97ad-8dee3a8b72b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:48.607548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:48.715605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:48.717746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:48.741976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 910/1822] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:48.745620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 913/1823] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:48.749564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.749971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.750434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.774708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:48.776749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:48.792875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 911/1824] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:48.797258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:48.808525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c924f7-7166-4237-87a0-cc37cb996c77 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:48.813666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:48.814282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.819093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.819949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.821019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:48.822184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:48.824707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:48.843282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 914/1825] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:48.843879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:48.860836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d92edd-aaf9-40b4-a01b-d8def97fae94 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:48.871950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:48.872846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:48.873323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:48.874257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:48.875121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:48.878299 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:48.956845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 912/1826] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:48 2022] POST /identity/v3/projects => generated 355 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:48.960947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:48.974912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6775f-ebdd-4005-9fba-996eae6ea108 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.005383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.008353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.071005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:49.129749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.130387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.131257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.131257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.131658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.134270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.171502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 915/1827] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:48 2022] POST /identity/v3/users => generated 278 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:49.175766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.184824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe547d-b1d1-4dea-9110-c69695e2c4fb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.200708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.201481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.201769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.202145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] PATH_INFO: `/v3/projects/856e161dc4cf435ebf1057a2881c0716` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.203767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=856e161dc4cf435ebf1057a2881c0716)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.206623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.210395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965cf94-4c89-4d7f-b23e-dd147e197727 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.229239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.234031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.234788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.234788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.235307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.237589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.276776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856e161dc4cf435ebf1057a2881c0716', 'request_id': 'req-9fa026d9-6f43-478c-8ae2-28995a0fa59a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.281016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 916/1828] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:49.286034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.288541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856e161dc4cf435ebf1057a2881c0716', 'request_id': 'req-9fa026d9-6f43-478c-8ae2-28995a0fa59a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.298850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856e161dc4cf435ebf1057a2881c0716', 'request_id': 'req-9fa026d9-6f43-478c-8ae2-28995a0fa59a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.314412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d9934-d159-4194-945e-977aa08a2736 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.319499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] The token cache is being invalidate because project 856e161dc4cf435ebf1057a2881c0716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:49.319801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.320736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.323524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 913/1829] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:49 2022] DELETE /identity/v3/projects/856e161dc4cf435ebf1057a2881c0716 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:49.328619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.330671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.367904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:49.423629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa026d9-6f43-478c-8ae2-28995a0fa59a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.432125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.432302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.432579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.432864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] PATH_INFO: `/v3/users/cce347a70d5a4181906ce2a5d6f12b91` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.434262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce347a70d5a4181906ce2a5d6f12b91)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.437124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.497790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.498851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.499987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.500309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.501024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.503103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.515816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 917/1830] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:49.526920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.532162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.546661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce347a70d5a4181906ce2a5d6f12b91'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.547945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce347a70d5a4181906ce2a5d6f12b91'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.550222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f943c-7f44-48a2-8657-120ea94d0348 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.557097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.557857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.558143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.558422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] PATH_INFO: `/v3/projects/445d73ace83e4fc8be9bb229fccce8cd/users/8ab01fbbf393453b9982036a0526d525/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.563294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce347a70d5a4181906ce2a5d6f12b91'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.564447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce347a70d5a4181906ce2a5d6f12b91'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.575718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ebbca1af-4292-4866-ae1e-06bcb6e5ccf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce347a70d5a4181906ce2a5d6f12b91'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:49.584019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445d73ace83e4fc8be9bb229fccce8cd, user_id=8ab01fbbf393453b9982036a0526d525, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.586919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.603527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 914/1831] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:49 2022] DELETE /identity/v3/users/cce347a70d5a4181906ce2a5d6f12b91 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:49.614755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 918/1832] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:49 2022] PUT /identity/v3/projects/445d73ace83e4fc8be9bb229fccce8cd/users/8ab01fbbf393453b9982036a0526d525/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:49.623590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.623884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.624173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.659984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] MFA Rules not processed for user `8ab01fbbf393453b9982036a0526d525`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:49.661610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.798258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 915/1833] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:49.802296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.805421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.889485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1d3fb-8a8c-4a96-a8f0-9a616aa36754 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.902487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:49.904144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.952520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:49.967866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:49.988441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.989143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.989497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.989846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.990566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.992737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.034537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 919/1834] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:49 2022] POST /identity/v3/projects => generated 357 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:50.036380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8b09b-3595-426c-adca-42e34d71b6f0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.038608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.156930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2ee2d-e9fc-4d67-9752-2b6e2cda3780 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.158513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.159315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.159485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.159741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] PATH_INFO: `/v3/users/305dde7a41194704865342335d92e970` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.161314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305dde7a41194704865342335d92e970)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.165695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.166888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.168106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.168967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.175043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.175977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.179903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.214400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 920/1835] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:50.219000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.223375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.237242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4598835-e8ee-4289-a61d-f447e6bc334c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.242269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.242961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.243239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.243518 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] PATH_INFO: `/v3/projects/b2c590404cd24993a3e7a7a22e36ab67` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.244699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c590404cd24993a3e7a7a22e36ab67)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.246687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.258754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305dde7a41194704865342335d92e970'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.260966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305dde7a41194704865342335d92e970'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.276292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305dde7a41194704865342335d92e970'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.277594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305dde7a41194704865342335d92e970'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.291208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305dde7a41194704865342335d92e970'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.322228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 916/1836] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:49 2022] DELETE /identity/v3/users/305dde7a41194704865342335d92e970 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:50.323231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.324650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.332733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c590404cd24993a3e7a7a22e36ab67', 'request_id': 'req-b04c7f63-d844-4163-ac6f-2d360d81dfca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.342252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c590404cd24993a3e7a7a22e36ab67', 'request_id': 'req-b04c7f63-d844-4163-ac6f-2d360d81dfca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.353642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c590404cd24993a3e7a7a22e36ab67', 'request_id': 'req-b04c7f63-d844-4163-ac6f-2d360d81dfca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.372933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] The token cache is being invalidate because project b2c590404cd24993a3e7a7a22e36ab67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:50.373537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.374231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b04c7f63-d844-4163-ac6f-2d360d81dfca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:50.376116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 921/1837] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:50 2022] DELETE /identity/v3/projects/b2c590404cd24993a3e7a7a22e36ab67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:50.381681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.381974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.382257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.408621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:50.409985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.421884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:50.431574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a65916-319b-4fad-a86f-dccc5db71cc9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.453869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.456111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.456525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.456808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.457479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.460597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7677f7-d68e-4b2b-847f-0360e70a4f0d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.468008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:50.497584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 917/1838] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:50.497584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6131-a200-4e1e-882b-aa23fcb0847d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.497584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6131-a200-4e1e-882b-aa23fcb0847d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.497584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6131-a200-4e1e-882b-aa23fcb0847d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.518347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72ef6131-a200-4e1e-882b-aa23fcb0847d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:50.519621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72ef6131-a200-4e1e-882b-aa23fcb0847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.519976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 922/1839] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:50.521009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.523239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.532508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 918/1840] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:50.537548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55a69e-9a4a-4573-adfa-7cb217cef1c9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.542271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.542527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.542912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.546896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.548049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.548434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.548768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] PATH_INFO: `/v3/projects/f0c330357d8c4e9098e7644c3688005d/users/d6e5809875b74eeb9467eb1aafecefbe/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.567173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:50.568903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.570947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c330357d8c4e9098e7644c3688005d, user_id=d6e5809875b74eeb9467eb1aafecefbe, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.574244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.578789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 919/1841] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:50.583366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.595128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 923/1842] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:50 2022] PUT /identity/v3/projects/f0c330357d8c4e9098e7644c3688005d/users/d6e5809875b74eeb9467eb1aafecefbe/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:50.595472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a293aa-9147-41be-9727-978a949eee4c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.596041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.598929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:50.670378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1d875-db56-408c-a5c1-6f7b800e7a22 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.675009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.675750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.676038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.676312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.730034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.731114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.731471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.731842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.733186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:50.733310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.735399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.751724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 920/1843] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:37:50.751724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.768388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f056784-752a-4f06-b9d1-fb714407305c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.778446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.780501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.782011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.782615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.784560 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.789450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.812222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 921/1844] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:50.813688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.832331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d921e-2c48-4efd-a35b-0e6bafb357db admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:50.840599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:50.841827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:50.854823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:50.854823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:50.858734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.862752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:50.865077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.866816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:50.898390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 924/1845] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:50.900807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.909220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 922/1846] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:50.914292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:50.932611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae3a0-276b-443a-95bd-adccc4a2947a service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.027293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802b51-fd7f-48eb-9a31-12f8e08bbcc7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.033095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.034425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.034782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.035201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.036003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.038224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.045215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.046848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.047532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.048093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] PATH_INFO: `/v3/projects/f0c330357d8c4e9098e7644c3688005d/users/d6e5809875b74eeb9467eb1aafecefbe/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.059084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c330357d8c4e9098e7644c3688005d, user_id=d6e5809875b74eeb9467eb1aafecefbe, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.061888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685896b0-211b-4ba4-9a74-90c210fa4cde admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.064347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:51.066965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 923/1847] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:50 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:51.071178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.086687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 925/1848] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:50 2022] PUT /identity/v3/projects/f0c330357d8c4e9098e7644c3688005d/users/d6e5809875b74eeb9467eb1aafecefbe/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:51.089659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518975-aa82-4c30-bc6b-93848e0131e6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.098764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.100151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.101696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.138559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] MFA Rules not processed for user `d6e5809875b74eeb9467eb1aafecefbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:51.140516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:51.166117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:51.268311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.268976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.269258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.269538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.270213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.271859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.293243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 926/1849] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:51.300962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 924/1850] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:51.302393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.308521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.310696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:51.327179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb4b21-5854-4cee-a531-502138cdb38f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.340102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.340739 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.341019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.341374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.342003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.347097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.380201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 927/1851] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:51.384717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.407872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90410b-a3f6-4dad-a793-70b06910beb9 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.420818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.421588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.421974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.422237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] PATH_INFO: `/v3/projects/3570f3625b2a418f883a60da4fec382d/users/1cef7d715204454c8214746dc1887b90/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.456166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3570f3625b2a418f883a60da4fec382d, user_id=1cef7d715204454c8214746dc1887b90, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.459642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bd79d-1033-4a19-99a0-c253174411e5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.468328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af46ba-6fa3-4bfa-a164-8aae919c7dfc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.492770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 928/1852] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:51 2022] PUT /identity/v3/projects/3570f3625b2a418f883a60da4fec382d/users/1cef7d715204454c8214746dc1887b90/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:51.495826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.496343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.496664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.553224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] MFA Rules not processed for user `1cef7d715204454c8214746dc1887b90`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:51.554861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:51.586744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.587378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.587657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.587930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.667888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 929/1853] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:51.669166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.734246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:51.781968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.785652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.787898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152a8b5-67d9-4569-803f-7ba5f9f0fb4f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.793921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:51.795059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:51.795658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:51.796084 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:51.799824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:51.803596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:51.833680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 925/1854] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:51.890455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 930/1855] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:51 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:51.893782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.913659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b31fe-c521-450b-974a-ebac308f6093 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:51.973280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:51.995695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:52.035647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.037280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.037280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.037280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.037605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.039766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.081736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 926/1856] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:51 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:52.086782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.109577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc2cf-e037-4bbb-a7fe-0f5069be2fef admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.112379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4600e52-63d2-4b8c-b280-f7d71dd18692 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.116839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.117538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.117752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.117873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.118098 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.118717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.118924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.119191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.119566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.120907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.124705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.149866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 927/1857] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:52.150917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.173245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413d9d-0b24-4925-bf96-eceec7a8d715 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.181390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.183620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.184075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.184507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] PATH_INFO: `/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0/users/0b5daf6bf0f241acbe07e7356b43acfe/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.212713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] RBAC: Authorizing `identity:create_grant(project_id=819d9eff0d3c48db9b4ba36ce3a146e0, user_id=0b5daf6bf0f241acbe07e7356b43acfe, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.215918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.250702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 928/1858] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:52 2022] PUT /identity/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0/users/0b5daf6bf0f241acbe07e7356b43acfe/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:52.251835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.253073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.259990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.277641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 931/1859] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:52.354795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.359549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.418673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a0a3-9663-449c-bc25-aadd3e0e1e30 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.490980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a27-9048-487b-a524-03b6a15f814c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.508147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.509647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.511057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.511356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.513088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.517128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.555640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 929/1860] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:52.567959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.576463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.577113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.577395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.577669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] PATH_INFO: `/v3/projects/4815cb1110cb41c193f0f622a6c02f46` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.578868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4815cb1110cb41c193f0f622a6c02f46)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.581655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.593697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e180b74-c3f9-4622-b6c4-7f11c6346a43 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.602964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:52.610340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.613100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.613981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.614269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] PATH_INFO: `/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0/users/0b5daf6bf0f241acbe07e7356b43acfe/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.621780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] RBAC: Authorizing `identity:create_grant(project_id=819d9eff0d3c48db9b4ba36ce3a146e0, user_id=0b5daf6bf0f241acbe07e7356b43acfe, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.630313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067dbeb-6818-4f0b-9ed6-6429972f096c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.645546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4815cb1110cb41c193f0f622a6c02f46', 'request_id': 'req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.645758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 930/1861] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:52 2022] PUT /identity/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0/users/0b5daf6bf0f241acbe07e7356b43acfe/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:52.653454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4815cb1110cb41c193f0f622a6c02f46', 'request_id': 'req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.657365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.657654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.657935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.662521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4815cb1110cb41c193f0f622a6c02f46', 'request_id': 'req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.680265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] The token cache is being invalidate because project 4815cb1110cb41c193f0f622a6c02f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:52.680843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.681901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.683704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 932/1862] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:52 2022] DELETE /identity/v3/projects/4815cb1110cb41c193f0f622a6c02f46 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:52.687619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.689278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.698962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] MFA Rules not processed for user `0b5daf6bf0f241acbe07e7356b43acfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:52.700240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.774601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f3820-1e70-4a34-82d8-9e084e3cfdd2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.784857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:52.797143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 931/1863] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:52.813398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.820059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.857785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.858362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.858614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.859006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] PATH_INFO: `/v3/users/d4fd6a57fbad449a8d309bba9d387a6f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.860339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fd6a57fbad449a8d309bba9d387a6f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.862314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.888001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08805f71-59d1-4569-807b-91d4c071ecfe None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.893242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.894058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.894366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.894625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.910000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.912702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.923547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 932/1864] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:52.927943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fd6a57fbad449a8d309bba9d387a6f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.928830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fd6a57fbad449a8d309bba9d387a6f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.931119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fd6a57fbad449a8d309bba9d387a6f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.936354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:52.943184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fd6a57fbad449a8d309bba9d387a6f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.961734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fd6a57fbad449a8d309bba9d387a6f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.976683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 933/1865] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:52 2022] DELETE /identity/v3/users/d4fd6a57fbad449a8d309bba9d387a6f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:53.015610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.025368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2715-ebbf-4ade-ba8e-c708a9cf325a service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.096248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.097186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.097475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.103043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.108160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.203240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.203709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.217919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 933/1866] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:53.477885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.480413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.647798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.652981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.695656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fee76d-4728-4681-9e85-2276575968ed admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.721605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.722276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.722675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.722955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] PATH_INFO: `/v3/projects/aa8d557059fa40b79f17a6aeee1db79d` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.737022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8d557059fa40b79f17a6aeee1db79d)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.745249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.782272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:53.819316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b10c4-ba5f-49e4-be23-0df4bfcbdf19 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.826986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.828975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.828975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.828975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] PATH_INFO: `/v3/users/93618779e43849ca95463e4d8d033655` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.829442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93618779e43849ca95463e4d8d033655)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.831583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.877944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8d557059fa40b79f17a6aeee1db79d', 'request_id': 'req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.894995 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8d557059fa40b79f17a6aeee1db79d', 'request_id': 'req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.906235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8d557059fa40b79f17a6aeee1db79d', 'request_id': 'req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.924497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] The token cache is being invalidate because project aa8d557059fa40b79f17a6aeee1db79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:53.925030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.926824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ea3b9693-2483-4be5-8a71-6f532c2f4b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.930440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 934/1867] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:53 2022] DELETE /identity/v3/projects/aa8d557059fa40b79f17a6aeee1db79d => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:53.935267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678911-4a87-4e0d-95f5-86c7200e637d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.935392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678911-4a87-4e0d-95f5-86c7200e637d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.935630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03678911-4a87-4e0d-95f5-86c7200e637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.980097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-03678911-4a87-4e0d-95f5-86c7200e637d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:53.982038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-03678911-4a87-4e0d-95f5-86c7200e637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.982511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93618779e43849ca95463e4d8d033655'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.983626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93618779e43849ca95463e4d8d033655'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.994036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93618779e43849ca95463e4d8d033655'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:53.995061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93618779e43849ca95463e4d8d033655'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:54.014554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca06b141-695e-423d-a8b7-954d403d0724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93618779e43849ca95463e4d8d033655'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:54.047039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 934/1868] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:53 2022] DELETE /identity/v3/users/93618779e43849ca95463e4d8d033655 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:54.058245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.058245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.058502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.089312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:54.092322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:54.136090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:54.166676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 935/1869] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:54.178146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.179558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.180314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.212275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:54.213844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:54.220740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 935/1870] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:54.229677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.230666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 936/1871] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:54.234512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.235232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:54.248100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2378c9-5953-43e0-a0cd-a0f0ef2facbf None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.254615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.255421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.255944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.256336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.257244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:54.257721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.260259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.286609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 937/1872] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:37:54.287618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.301100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ab626-1d50-43c1-aba5-56dbab64d90d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.309685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.310483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.311492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.311779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.312851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.315848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.355945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f422524-610f-4999-a29b-f85e9ef00607 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.358611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 938/1873] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:54.365193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.394671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:54.447782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.448424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.448765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.449050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.485676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931566d5-acff-4c26-bf01-6ab7780f8339 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.491197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.491871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.492281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.492567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.493214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.495410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.526976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 939/1874] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:54.528305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.546680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593601b-24c6-415f-bee1-3b501a2385e0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.546680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.546680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.546680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.547104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.547104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.547250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.547343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.547862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.558437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 940/1875] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:54.559808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 936/1876] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:54.562505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.572875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d44352-5097-41d8-affb-319cdf084644 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.577846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.578440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.578809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.579085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.579838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.581484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d5e58-b412-4aad-bf62-9cd740dbf21e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.592167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 941/1877] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:37:54.596681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.606795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c236bd-0d4a-4227-b6e9-03db3ca74dcd service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.612843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.613510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.613996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.614115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] PATH_INFO: `/v3/projects/c14401bfa166434fa5b95cadbb3c3622/users/8be2c8fb26274052bede9e98226c671c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.634515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14401bfa166434fa5b95cadbb3c3622, user_id=8be2c8fb26274052bede9e98226c671c, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.637545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.678754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 937/1878] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:54 2022] PUT /identity/v3/projects/c14401bfa166434fa5b95cadbb3c3622/users/8be2c8fb26274052bede9e98226c671c/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:54.681655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.681969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.682430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.725781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] MFA Rules not processed for user `8be2c8fb26274052bede9e98226c671c`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:54.728199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:54.814939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 942/1879] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:54.820112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.916083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28748423-2886-4a83-a6a7-ceadd58eb7eb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:54.927920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:54.928432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:54.928712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:54.929051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:54.929696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:54.932000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:54.971559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:54.973606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:54.977864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 938/1880] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:54.979168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.010561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b5159-b893-4a4a-b534-bd9692b725b8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.052384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:55.060404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:55.093337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed3b7b-698c-4774-917a-9fc8c5aef5ac admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.100817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.101448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.101732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.102005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.102797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.106872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.143166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 939/1881] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:54 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:55.146535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.170058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cba10-4e1a-420f-aedc-e9cb5b2274c9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.197029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.197896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.198718 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.199213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.199493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.199849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.200208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] PATH_INFO: `/v3/projects/4498b9ca612d47f7b8f3d8fc34028640` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.211550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4498b9ca612d47f7b8f3d8fc34028640)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.211550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.226965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.227803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.229449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.262442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 940/1882] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:55.268634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.295005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050658f-6f3f-4c17-a214-e3bd31516a0f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.308592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.309915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.310998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.311870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] PATH_INFO: `/v3/projects/35af47790863406e89a02603608d08f2/users/46703d13a5704971a6707b0d3214e64c/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.341175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4498b9ca612d47f7b8f3d8fc34028640', 'request_id': 'req-bc6bdfb9-be34-491b-ac5f-4512108a6e41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.350907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35af47790863406e89a02603608d08f2, user_id=46703d13a5704971a6707b0d3214e64c, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.354472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.367610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4498b9ca612d47f7b8f3d8fc34028640', 'request_id': 'req-bc6bdfb9-be34-491b-ac5f-4512108a6e41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.375673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4498b9ca612d47f7b8f3d8fc34028640', 'request_id': 'req-bc6bdfb9-be34-491b-ac5f-4512108a6e41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.390827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 941/1883] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:55 2022] PUT /identity/v3/projects/35af47790863406e89a02603608d08f2/users/46703d13a5704971a6707b0d3214e64c/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:55.395086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.407815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] The token cache is being invalidate because project 4498b9ca612d47f7b8f3d8fc34028640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:55.408095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.409833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.414937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 943/1884] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:54 2022] DELETE /identity/v3/projects/4498b9ca612d47f7b8f3d8fc34028640 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:55.419842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.421573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:55.566263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef8663-c77e-4d7a-b67a-c2fa99da2046 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.573852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.574822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.575277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.575657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.576674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.577258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bdfb9-be34-491b-ac5f-4512108a6e41 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.579219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.592166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.592837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.593119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.593395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] PATH_INFO: `/v3/users/0c05a01601834f729e7f202261030927` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.593736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 942/1885] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:55.598121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c05a01601834f729e7f202261030927)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.600141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.600825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.603947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:55.634987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb1301-592a-4d8b-b40d-bcc7c7047bea admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:55.644749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:55.645572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.645908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.646309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] PATH_INFO: `/v3/projects/35af47790863406e89a02603608d08f2/users/46703d13a5704971a6707b0d3214e64c/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.652769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35af47790863406e89a02603608d08f2, user_id=46703d13a5704971a6707b0d3214e64c, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:55.656041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba105453-bd69-460d-b5c9-a97cc0fecd3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:55.709167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 943/1886] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:55 2022] PUT /identity/v3/projects/35af47790863406e89a02603608d08f2/users/46703d13a5704971a6707b0d3214e64c/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:55.711204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb1a2-6e51-4b00-b950-5eb4da11ae83 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:55.711204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb1a2-6e51-4b00-b950-5eb4da11ae83 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:55.711423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb1a2-6e51-4b00-b950-5eb4da11ae83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:55.718818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c05a01601834f729e7f202261030927'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.719646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c05a01601834f729e7f202261030927'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.721506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c05a01601834f729e7f202261030927'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.723446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:55.739934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c05a01601834f729e7f202261030927'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.768392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c05a01601834f729e7f202261030927'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:55.796184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-41aeb1a2-6e51-4b00-b950-5eb4da11ae83 None None] MFA Rules not processed for user `46703d13a5704971a6707b0d3214e64c`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:55.797453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-41aeb1a2-6e51-4b00-b950-5eb4da11ae83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:55.803410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 944/1887] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:55 2022] DELETE /identity/v3/users/0c05a01601834f729e7f202261030927 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:55.909763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:55.915065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 944/1888] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:55.929100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:55.930886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:56.022825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1e78b-3951-4df0-b62b-89f5235ee02b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:56.031714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:56.032669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:56.034112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:56.034891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:56.141229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:56.144280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46dc76-a542-4072-b86f-e546c0263a9d service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:56.170368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 945/1889] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:56.562279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:56.562846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:56.563372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:56.565247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 945/1890] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:37:56.580533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:56.581087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:56.581164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:56.625520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:56.626993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:56.705059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 946/1891] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:37:56.824065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:56.827023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:56.933458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9210f-960c-417b-bb12-71e6a5fb252f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:56.939770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:56.940445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:56.940785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:56.941075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] PATH_INFO: `/v3/users/1cef7d715204454c8214746dc1887b90` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:56.942959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cef7d715204454c8214746dc1887b90)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:56.944796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.053587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.055562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.087080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472cc6d-492c-49c0-a306-2babce093f8b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.089347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cef7d715204454c8214746dc1887b90'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.092014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cef7d715204454c8214746dc1887b90'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.124116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cef7d715204454c8214746dc1887b90'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.124406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.125030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.125305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.125578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.127252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cef7d715204454c8214746dc1887b90'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.127954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.147505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cef7d715204454c8214746dc1887b90'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.160010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:57.167772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.170114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6acbd-dc06-4c7f-bc01-2eb116e4c122 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.197816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 947/1892] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:57.257740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 946/1893] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:56 2022] DELETE /identity/v3/users/1cef7d715204454c8214746dc1887b90 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:57.262051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.262912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.263096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.271472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.273481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.306549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:57.308328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.426276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 948/1894] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:57.427580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.429116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.437652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f37f-62f7-4bc9-8dad-7e9b4e292c87 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.443068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.443489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605cab0-f29c-43bc-a1f1-d73356a968a9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.443715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.444145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.444428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] PATH_INFO: `/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.445993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee6e4e9af8b48f582e3baa9dccb820a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.448375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.449422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.450226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.450689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.451137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] PATH_INFO: `/v3/users/8be2c8fb26274052bede9e98226c671c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.455386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be2c8fb26274052bede9e98226c671c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.457681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.484901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:57.535056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee6e4e9af8b48f582e3baa9dccb820a', 'request_id': 'req-e960d6bf-7a4c-466e-b456-3527f637b478'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.535178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be2c8fb26274052bede9e98226c671c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.535178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be2c8fb26274052bede9e98226c671c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.535178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be2c8fb26274052bede9e98226c671c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.539473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be2c8fb26274052bede9e98226c671c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.541254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee6e4e9af8b48f582e3baa9dccb820a', 'request_id': 'req-e960d6bf-7a4c-466e-b456-3527f637b478'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.547943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee6e4e9af8b48f582e3baa9dccb820a', 'request_id': 'req-e960d6bf-7a4c-466e-b456-3527f637b478'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.563432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-89abcf2e-2db1-4b57-a5fc-560e8a1e281d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be2c8fb26274052bede9e98226c671c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.567027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] The token cache is being invalidate because project 9ee6e4e9af8b48f582e3baa9dccb820a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:57.567546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.568101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:57.569869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 947/1895] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:57 2022] DELETE /identity/v3/projects/9ee6e4e9af8b48f582e3baa9dccb820a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:57.570441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.571806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.588508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 949/1896] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:57 2022] DELETE /identity/v3/users/8be2c8fb26274052bede9e98226c671c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:57.589924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b28e08-3a25-480c-9358-f95cf3a616ee None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.590164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b28e08-3a25-480c-9358-f95cf3a616ee None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.590594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b28e08-3a25-480c-9358-f95cf3a616ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.633315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-06b28e08-3a25-480c-9358-f95cf3a616ee None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:57.634578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-06b28e08-3a25-480c-9358-f95cf3a616ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.647099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960d6bf-7a4c-466e-b456-3527f637b478 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.701736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.702360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.702722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.702989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.704716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.733032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 950/1897] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:57.755273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc4bb-8dce-4d05-999e-3bc4ccfc54c5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.755593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc4bb-8dce-4d05-999e-3bc4ccfc54c5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.755918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dc4bb-8dce-4d05-999e-3bc4ccfc54c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.779403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:57.793451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-cc1dc4bb-8dce-4d05-999e-3bc4ccfc54c5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:57.794846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cc1dc4bb-8dce-4d05-999e-3bc4ccfc54c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.805768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 951/1898] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:57.810867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.811271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.811566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.833660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:57.836376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.844145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 952/1899] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:57.845110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.847534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.848909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.859218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3a743-eb1e-478b-a412-52de592c3062 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.865925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 948/1900] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:57.880779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.881856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.882321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.882873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.893382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.901996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.903919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.919951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 953/1901] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:57.925000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a961f1-5b19-4adf-9d73-91daed678ce7 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.935930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.936556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.936839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.937121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.937758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:57.938060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.939774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.955654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 949/1902] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:57.960189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.978294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc0e86-9a8e-4f76-b1f2-ccf96fd99b7f service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.987811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.989795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.990303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.990853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.992694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.995875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.057279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 954/1903] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:58.065948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.073855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:58.083807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbb0bf-faa8-4b7a-8b67-26acce169e6c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.166913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.168431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.169320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.169719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.170458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.172595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.208498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 950/1904] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:58 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:58.209290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.222033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11325d74-0026-4f08-b0b4-45bec7914bb6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.229484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.230113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.230458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.230809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.231530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.233252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.260433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 955/1905] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:58.264909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.275630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b201-6c01-4ead-9149-9167da60519e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.280753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.281514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.281625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.281904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.282559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.285133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c524f9-a178-4753-9373-776261dddc06 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.303038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 951/1906] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:58.303819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.319247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7f60-7d29-4d33-8006-34e7ece5c6f3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.326299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.327328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.327768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.328154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] PATH_INFO: `/v3/projects/14d8969c821f450b9167d782823f8c12/users/806523a2c38e428cbc6cbf4cb058514f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.351893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d8969c821f450b9167d782823f8c12, user_id=806523a2c38e428cbc6cbf4cb058514f, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.355767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.379006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 956/1907] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:58 2022] PUT /identity/v3/projects/14d8969c821f450b9167d782823f8c12/users/806523a2c38e428cbc6cbf4cb058514f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:58.389383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.389677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.389953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.453305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] MFA Rules not processed for user `806523a2c38e428cbc6cbf4cb058514f`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:58.454999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:58.627362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 952/1908] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:58.639416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.641966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:58.724490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.728181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:58.769157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade50-8778-45ae-86e4-36b9c6015271 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.778207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.779345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.779821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.780320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.797701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.800524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.845533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 957/1909] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:58.848122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:58.851298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:58.948913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbae0c-bd47-41f6-b46c-7dd7a02bec19 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:58.956055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:58.957681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:58.957975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:58.958257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] PATH_INFO: `/v3/projects/c14401bfa166434fa5b95cadbb3c3622` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:58.959521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14401bfa166434fa5b95cadbb3c3622)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:58.962936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:58.981328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:59.008428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e395b-4095-4129-9ccb-cf8408a0bfd1 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.016753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.017414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.017907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.018292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] PATH_INFO: `/v3/projects/3570f3625b2a418f883a60da4fec382d` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.019557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3570f3625b2a418f883a60da4fec382d)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.021542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.039876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14401bfa166434fa5b95cadbb3c3622', 'request_id': 'req-f17f49a0-55ea-4ebc-baec-927f4cd4766b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.047152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14401bfa166434fa5b95cadbb3c3622', 'request_id': 'req-f17f49a0-55ea-4ebc-baec-927f4cd4766b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.066756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14401bfa166434fa5b95cadbb3c3622', 'request_id': 'req-f17f49a0-55ea-4ebc-baec-927f4cd4766b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.080033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:59.083714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] The token cache is being invalidate because project c14401bfa166434fa5b95cadbb3c3622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:59.084077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.084726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.086569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 953/1910] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:58 2022] DELETE /identity/v3/projects/c14401bfa166434fa5b95cadbb3c3622 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:59.087749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:59.091398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.106934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3570f3625b2a418f883a60da4fec382d', 'request_id': 'req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.114474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3570f3625b2a418f883a60da4fec382d', 'request_id': 'req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.121418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3570f3625b2a418f883a60da4fec382d', 'request_id': 'req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.138664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] The token cache is being invalidate because project 3570f3625b2a418f883a60da4fec382d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:59.138968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.139618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.141308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 958/1911] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:58 2022] DELETE /identity/v3/projects/3570f3625b2a418f883a60da4fec382d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:59.142043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:59.143666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.179283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f49a0-55ea-4ebc-baec-927f4cd4766b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.229155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dd7cd-3534-4dcf-85d2-73e51473fc70 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.235205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.236745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.237135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.237416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] PATH_INFO: `/v3/users/46703d13a5704971a6707b0d3214e64c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.238795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=46703d13a5704971a6707b0d3214e64c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.240694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.243719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.244357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.244647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.244919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] PATH_INFO: `/v3/users/806523a2c38e428cbc6cbf4cb058514f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.246099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806523a2c38e428cbc6cbf4cb058514f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.248065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.288788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46703d13a5704971a6707b0d3214e64c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.289388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46703d13a5704971a6707b0d3214e64c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.290086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46703d13a5704971a6707b0d3214e64c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.295073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806523a2c38e428cbc6cbf4cb058514f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.295893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806523a2c38e428cbc6cbf4cb058514f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.297974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:59.298109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46703d13a5704971a6707b0d3214e64c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.305501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806523a2c38e428cbc6cbf4cb058514f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.306819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806523a2c38e428cbc6cbf4cb058514f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.312383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46703d13a5704971a6707b0d3214e64c'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.318556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fada0afe-102e-486f-9fbe-8a83495e5055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806523a2c38e428cbc6cbf4cb058514f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.328403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 959/1912] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:59 2022] DELETE /identity/v3/users/46703d13a5704971a6707b0d3214e64c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:59.329406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:59.331047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.343504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 954/1913] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:59 2022] DELETE /identity/v3/users/806523a2c38e428cbc6cbf4cb058514f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:59.349899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.350300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.350794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.377930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:59.379378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.382428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:59.420818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4d719-0e5f-4ba0-86ff-72afc10e79ca admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.496634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 955/1914] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:59.508369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:59.510012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.511417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.512040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.512329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.512667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] PATH_INFO: `/v3/users/0b5daf6bf0f241acbe07e7356b43acfe` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.514059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5daf6bf0f241acbe07e7356b43acfe)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.516971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.612436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0b74-4b6b-4d33-aa7e-b5ce96a13f7e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.619195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.619195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.619195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.619487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.634270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5daf6bf0f241acbe07e7356b43acfe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.635106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5daf6bf0f241acbe07e7356b43acfe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.636184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5daf6bf0f241acbe07e7356b43acfe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.639530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.645460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.654369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5daf6bf0f241acbe07e7356b43acfe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.671309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 956/1915] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:37:59.679737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5daf6bf0f241acbe07e7356b43acfe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:59.695470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 960/1916] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:59 2022] DELETE /identity/v3/users/0b5daf6bf0f241acbe07e7356b43acfe => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:00.013859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:00.016128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.034114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8973-0e24-44db-bb25-c8243b838775 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:00.101577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:00.103212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.103212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.103212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] PATH_INFO: `/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.104169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819d9eff0d3c48db9b4ba36ce3a146e0)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:00.106331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:00.123201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:00.209067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819d9eff0d3c48db9b4ba36ce3a146e0', 'request_id': 'req-1e0989d6-0dc0-456d-95c1-456417aaf622'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.216703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819d9eff0d3c48db9b4ba36ce3a146e0', 'request_id': 'req-1e0989d6-0dc0-456d-95c1-456417aaf622'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.224436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819d9eff0d3c48db9b4ba36ce3a146e0', 'request_id': 'req-1e0989d6-0dc0-456d-95c1-456417aaf622'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.239986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] The token cache is being invalidate because project 819d9eff0d3c48db9b4ba36ce3a146e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:00.240601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.241551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e0989d6-0dc0-456d-95c1-456417aaf622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.244021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 957/1917] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:00 2022] DELETE /identity/v3/projects/819d9eff0d3c48db9b4ba36ce3a146e0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:00.250309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:00.251428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5306-2bbc-4111-8289-9c835e12ba84 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.252100 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5306-2bbc-4111-8289-9c835e12ba84 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.252752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5306-2bbc-4111-8289-9c835e12ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.253374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.276797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2cf3-4dec-4b36-a030-a465416e2fea admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:00.277620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-78bb5306-2bbc-4111-8289-9c835e12ba84 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:00.279258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-78bb5306-2bbc-4111-8289-9c835e12ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.420600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 958/1918] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:00.426696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:00.427737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.428027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.428520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] PATH_INFO: `/v3/projects/35af47790863406e89a02603608d08f2` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.429828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35af47790863406e89a02603608d08f2)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:00.430535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.431076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.431793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.435457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:00.440982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:00.489975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:00.491318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.500576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 959/1919] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:00.505214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:00.521413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b3ce8-81d8-45df-8c04-bbc509188e54 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:00.557051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35af47790863406e89a02603608d08f2', 'request_id': 'req-27c107d1-0e25-4c53-a30c-92490f2d219d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.568085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35af47790863406e89a02603608d08f2', 'request_id': 'req-27c107d1-0e25-4c53-a30c-92490f2d219d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.593471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35af47790863406e89a02603608d08f2', 'request_id': 'req-27c107d1-0e25-4c53-a30c-92490f2d219d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.609873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:00.610665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.611700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.612253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.613105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:00.613527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:00.615287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:00.630243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 960/1920] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:38:00.639223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:00.644817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] The token cache is being invalidate because project 35af47790863406e89a02603608d08f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:00.645447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.652263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-27c107d1-0e25-4c53-a30c-92490f2d219d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:00.658302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 961/1921] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:00 2022] DELETE /identity/v3/projects/35af47790863406e89a02603608d08f2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:00.678749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b56e4-fbc5-4e55-ae72-fd2912d7bf3a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:00.680867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.681278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.683032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.686825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:00.687809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.688218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.689334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.690439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:00.693640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:00.770857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:00.800065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 961/1922] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:00 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:00.801142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:00.805431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.806480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:00.839056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:00.927277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e658b3-a0bd-460a-bf2e-1c9f32b8a239 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:00.934187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:00.934886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:00.935196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:00.935470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:00.936969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:00.939581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.000691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 962/1923] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:00 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:38:01.003482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.008764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.045135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39f-4908-43f4-980b-f6883e630bdf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.052665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.053409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.053765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.055244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] PATH_INFO: `/v3/projects/14d8969c821f450b9167d782823f8c12` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.057399 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d8969c821f450b9167d782823f8c12)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.061629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.092742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 962/1924] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:01.093968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.115308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db2bfc-e8a8-4c40-bf10-1db03fc10cf2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.127436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.129231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.129521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.129802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.131028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.135444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d114513-87ed-4bf0-af27-d91a4b84b8bd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.150353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d8969c821f450b9167d782823f8c12', 'request_id': 'req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:01.157002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 963/1925] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:01.159412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d8969c821f450b9167d782823f8c12', 'request_id': 'req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:01.159656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ecd6-3965-4b71-9989-9d2ae3a1877b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.160049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ecd6-3965-4b71-9989-9d2ae3a1877b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.160441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ecd6-3965-4b71-9989-9d2ae3a1877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.167282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d8969c821f450b9167d782823f8c12', 'request_id': 'req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:01.183891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] The token cache is being invalidate because project 14d8969c821f450b9167d782823f8c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:01.184188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:01.184624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5f61ecd6-3965-4b71-9989-9d2ae3a1877b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:01.184855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:01.185869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5f61ecd6-3965-4b71-9989-9d2ae3a1877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.186442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 963/1926] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:01 2022] DELETE /identity/v3/projects/14d8969c821f450b9167d782823f8c12 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:01.187329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.188639 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.256899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 964/1927] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:01.257954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.258245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.258524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.268251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b2f9-5979-40ad-b019-ec9c9cf61f01 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.277433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.278382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.278881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.279332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.280159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.281829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:01.283381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da380c2c-186a-4d09-a8dd-4af60fb1901f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.283721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.292442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 965/1928] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:01.293410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.305665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 964/1929] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:01.305942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d28b4-63c3-491c-8569-cc5e3588a886 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.307859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.307859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.308124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.314390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.315287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.315817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.316162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.316927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:01.317299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.319105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.334308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 966/1930] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:01.336298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:01.337859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.338375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.344189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:01.349143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 965/1931] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:01.350028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.354395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6caf-ec88-4f9b-9f65-550e9e18f065 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.361250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.361925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.362208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.362486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] PATH_INFO: `/v3/projects/a167db5e9c164070a21feb916e1906cb/users/0e295e00b2844af282b31873a8612871/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.363596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b166d5-e606-424b-b1b7-4685213a81d3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.374358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.375357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.375827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.376284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.377178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.385045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb7ac2-f7dd-4bd6-be10-d15a163f50ae admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.390988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a167db5e9c164070a21feb916e1906cb, user_id=0e295e00b2844af282b31873a8612871, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.392545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:01.396424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.416969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 967/1932] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:01 2022] PUT /identity/v3/projects/a167db5e9c164070a21feb916e1906cb/users/0e295e00b2844af282b31873a8612871/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:01.418241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.437838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 966/1933] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.437838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.437838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.437838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.468386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] MFA Rules not processed for user `0e295e00b2844af282b31873a8612871`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:01.469701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.508121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f82-56c1-4fd2-9887-cbe96cdc42e7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.550121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 967/1934] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:01.554336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.603361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.603985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.604275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.604599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.605267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:01.605634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.607383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.619494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 968/1935] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:01.620380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.631240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da150a9e-d96a-4a8c-8d65-41d8f93dd9ba admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.635699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.636286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.636574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.636848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.637496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.638599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fd587-2424-40c9-b7a8-724e22ea80ce None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.639407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.644333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.644939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.645220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.645497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.646159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.648179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.670828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 969/1936] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.671483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 968/1937] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.671599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.672651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.682424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456af59-1b2b-4509-b302-c9a09dd83ce0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.732381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:01.736648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c0e5f-91e3-41e8-9c77-fbb96e135136 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.740338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.740956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.741250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.741510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.742144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.742509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.743144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.743425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.743698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.744124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.744318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.746197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.756042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:01.768220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 969/1938] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.769185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.778329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 970/1939] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.778917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b54c8a-f9bd-420e-a3bb-71c65470b884 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.779028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.838319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.838979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.839258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.839531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.840183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.841812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.843462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bbb35-d3c0-49bd-a527-5e2c2ad92819 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.848537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.849145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.849427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.849701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.850350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.852012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.852416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 970/1940] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:01.856376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.862487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 971/1941] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:01.863338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.867304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3eac4-1f08-48b4-88f1-7c065b22ca35 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.872257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.872846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.872987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2655d-ab66-496f-8501-08809c69dd9f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.873131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.873414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.874037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.875971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.878617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.879247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.879532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.879810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.880491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.882241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.894195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 972/1942] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:01.895270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.903438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 971/1943] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:01 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:01.904772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.907802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3bf1-4f6d-4c46-95ec-c9166e262b93 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.913295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.913668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.913974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.914253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] PATH_INFO: `/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b/users/0d4fadbfd2ee4261b52bf2998b9cdd08/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.915561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc216159-2eaf-4ba7-9931-a484cc69f0f1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.925710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.927493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.929050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.929585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] PATH_INFO: `/v3/projects/80c17e53479e4f88b363de447a5f932b/users/b8340a5a706b4eaa984c5758351632ac/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.933850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a92494eaa8e4edab4d556b5b9d1c98b, user_id=0d4fadbfd2ee4261b52bf2998b9cdd08, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.937073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.948496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c17e53479e4f88b363de447a5f932b, user_id=b8340a5a706b4eaa984c5758351632ac, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.952287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.958053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 973/1944] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:01 2022] PUT /identity/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b/users/0d4fadbfd2ee4261b52bf2998b9cdd08/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:01.959094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.977467 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 972/1945] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:01 2022] PUT /identity/v3/projects/80c17e53479e4f88b363de447a5f932b/users/b8340a5a706b4eaa984c5758351632ac/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:01.978905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.059146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4881c5-c8fa-49e2-bf7c-0f27dec4d181 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.061964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc116b0-e462-48aa-b3d2-b7fe6eac1bee admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.065510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.066135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.066539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.066960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.067853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.068589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.069330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.069681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.069988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465dc1b-adea-46b2-922f-36d6b7440e22 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.070267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.071126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.073129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.089906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 974/1946] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:02.092611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 973/1947] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:02.096606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.096910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.097068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.097177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.127331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda493d8-0ed9-4a39-a840-682427a1c5c9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.135798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.136287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.136638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.136922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.137945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.139060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:02.140134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.142681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.168391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 975/1948] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:02.172932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 974/1949] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:02.174479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.176250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.193729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98931df-a474-4ade-b94c-f43a67ccf032 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.199244 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.199595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9f90c-5274-4908-82ac-19f5ab6e043b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.199918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.200203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.200467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] PATH_INFO: `/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b/users/0d4fadbfd2ee4261b52bf2998b9cdd08/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.202435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a92494eaa8e4edab4d556b5b9d1c98b, user_id=0d4fadbfd2ee4261b52bf2998b9cdd08, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.205566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.216689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.217702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.218473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.220743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] PATH_INFO: `/v3/projects/cf4f846304ca417597295c3077cfe197/users/773141e01d2c4813af90305d44a9db41/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.227054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 976/1950] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:02 2022] PUT /identity/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b/users/0d4fadbfd2ee4261b52bf2998b9cdd08/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:02.228061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.230301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.254893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4f846304ca417597295c3077cfe197, user_id=773141e01d2c4813af90305d44a9db41, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.259053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee6b66-eb57-4fd8-b78b-094314bd0f11 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.286676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 975/1951] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:02 2022] PUT /identity/v3/projects/cf4f846304ca417597295c3077cfe197/users/773141e01d2c4813af90305d44a9db41/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:02.288227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98b20-4a13-4f21-ae90-8f0e907fcdc6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.290683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98b20-4a13-4f21-ae90-8f0e907fcdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.296411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e98b20-4a13-4f21-ae90-8f0e907fcdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.302083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:02.321725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:02.334248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d0e98b20-4a13-4f21-ae90-8f0e907fcdc6 None None] MFA Rules not processed for user `0d4fadbfd2ee4261b52bf2998b9cdd08`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:02.335540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d0e98b20-4a13-4f21-ae90-8f0e907fcdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.381595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc46114-63a5-4cef-af7c-aee380748964 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.425297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 976/1952] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:02.431111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.431291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.431568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.459269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.460158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.460583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.461068 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.481907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] MFA Rules not processed for user `773141e01d2c4813af90305d44a9db41`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:02.483196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.546327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.548116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.558773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 977/1953] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:02.559551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.561020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.604233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 977/1954] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:02.646552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8396a-01f0-4b1d-84dd-b7e41acfa265 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.653618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.653618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.653909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.654329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.690291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:02.694733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:02.719764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:02.721597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc1e78-6eb6-4759-a076-7bd0c734088a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:02.727419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:02.728470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:02.728821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:02.729162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:02.745308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.747020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb24eec-7e22-4ed8-9330-1453d200cce6 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.758074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:02.759790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:02.762934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 978/1955] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:02.787456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 978/1956] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:03.820909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:03.821501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:03.821501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:03.901636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] MFA Rules not processed for user `b8340a5a706b4eaa984c5758351632ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:03.904949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:04.038947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 979/1957] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:04.043766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.063302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8944081-bc19-4323-ad98-0f373f0b2828 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.068865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.069535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.072020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.072408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.074783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.077398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.134765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.143324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:04.146155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 979/1958] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:04 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:04.150564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c5de-7a18-4f32-8eba-9311587c1ce3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.226337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.226984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.227268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.227628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.228186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.230107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.256363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 980/1959] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:04 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:04.257947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.269423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ecae-e243-41a2-9d73-182bd3e8456d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.274147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.274793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.275079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.275352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.276002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.277595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.289686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 980/1960] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:04.294019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.304321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f0bb-aa20-42c7-875e-10d697042066 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.308891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.309542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.309819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.310120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] PATH_INFO: `/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948/users/6ad19af637bb400eafd18a6709ab350e/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.331675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26de7a84fcd34fcb8b0c2e9bf5d1e948, user_id=6ad19af637bb400eafd18a6709ab350e, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.334297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.350106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 981/1961] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:04 2022] PUT /identity/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948/users/6ad19af637bb400eafd18a6709ab350e/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:04.354358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.364556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673ef66-53cc-4cbc-8e85-d8ec214d2af0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.421851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.422470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.422789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.423071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.424047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.425704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bbadf-3b5a-4690-a57a-3388ce12de93 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.437623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 981/1962] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:04.442683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.452842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470198c-d11c-4feb-90b6-98e194257d4b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.459137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.460926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.461327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.461498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] PATH_INFO: `/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948/users/6ad19af637bb400eafd18a6709ab350e/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.463498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26de7a84fcd34fcb8b0c2e9bf5d1e948, user_id=6ad19af637bb400eafd18a6709ab350e, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.466208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.477550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 982/1963] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:04 2022] PUT /identity/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948/users/6ad19af637bb400eafd18a6709ab350e/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:05.442768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:05.443214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:05.443384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:05.493547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] MFA Rules not processed for user `6ad19af637bb400eafd18a6709ab350e`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:05.494728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:05.604539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 982/1964] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:05.612813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:05.675816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278af9b-f0c8-434a-b0fa-d4bc788abce8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:05.679534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:05.680211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:05.680566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:05.680908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:05.740096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:05.741546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:05.752690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 983/1965] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:08.885922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:08.897086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc1a07-7bba-4c44-9a58-a4514134a101 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:08.900956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:08.901587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:08.901897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:08.902170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:08.929328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:08.967090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:08.968529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:08.978540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 983/1966] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:15.664304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:15.673448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eb974-e186-491f-afc2-1b66ddb44e63 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:15.677875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:15.678761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:15.679050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:15.679325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:15.687110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:15.690479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:15.692185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:15.703706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 984/1967] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:17.432253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:17.524240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79478-39eb-49f0-972f-2ea575334432 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:17.531184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:17.532089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.532637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.533186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] PATH_INFO: `/v3/users/773141e01d2c4813af90305d44a9db41` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.534572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=773141e01d2c4813af90305d44a9db41)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:17.536587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:17.569794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:17.571395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:17.588637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe75bb-575c-44d0-b657-7dccd51933b5 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:17.593350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:17.593976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.594260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.594770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] PATH_INFO: `/v3/users/8ab01fbbf393453b9982036a0526d525` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.596644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab01fbbf393453b9982036a0526d525)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:17.599375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:17.600220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '773141e01d2c4813af90305d44a9db41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.600913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '773141e01d2c4813af90305d44a9db41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.601531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '773141e01d2c4813af90305d44a9db41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.610827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '773141e01d2c4813af90305d44a9db41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.624304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ee9b5987-b45c-4670-a626-278c8b0f9d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '773141e01d2c4813af90305d44a9db41'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.638896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 984/1968] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:17 2022] DELETE /identity/v3/users/773141e01d2c4813af90305d44a9db41 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:17.643999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.644258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.644537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.654951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab01fbbf393453b9982036a0526d525'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.655617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab01fbbf393453b9982036a0526d525'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.663060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab01fbbf393453b9982036a0526d525'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.663749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab01fbbf393453b9982036a0526d525'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.664404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:17.665769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:17.671603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-864195fe-39eb-456a-9ce5-0da4bc4cda27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab01fbbf393453b9982036a0526d525'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:17.692823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 985/1969] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:17 2022] DELETE /identity/v3/users/8ab01fbbf393453b9982036a0526d525 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:17.697246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.697537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.697813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.705888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:17.719647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:17.720862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:17.729584 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 985/1970] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:17.740538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:17.741943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:17.784115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 986/1971] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:38:17.793868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:17.801547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e48f75-d19c-42cb-a96f-e6cd19d55818 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:17.849883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1ccf-7ee9-4076-8345-20e48db5fd97 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:17.852291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:17.852902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.853183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.853471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:17.853576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.854076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:17.854357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:17.854623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:17.863575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:17.864683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:17.865521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:17.866200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:17.876908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 987/1972] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:38:17.877223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 986/1973] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:18.428918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.430375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.435173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.436624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.445277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd627-d99b-471f-9deb-300631c34970 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.451220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd50d67-b26a-451b-b0e0-3c123556ee5d service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.453825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.455209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.456169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.456670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] PATH_INFO: `/v3/projects/cf4f846304ca417597295c3077cfe197` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.458438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.458709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4f846304ca417597295c3077cfe197)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.459292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.459646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.460033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] PATH_INFO: `/v3/projects/445d73ace83e4fc8be9bb229fccce8cd` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.460733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.461433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445d73ace83e4fc8be9bb229fccce8cd)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.463828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.469191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:18.532791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445d73ace83e4fc8be9bb229fccce8cd', 'request_id': 'req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.536963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4f846304ca417597295c3077cfe197', 'request_id': 'req-5f1caa4c-562c-43b4-836f-4788c57fcd0e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.539918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445d73ace83e4fc8be9bb229fccce8cd', 'request_id': 'req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.543677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4f846304ca417597295c3077cfe197', 'request_id': 'req-5f1caa4c-562c-43b4-836f-4788c57fcd0e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.546835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445d73ace83e4fc8be9bb229fccce8cd', 'request_id': 'req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.551453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4f846304ca417597295c3077cfe197', 'request_id': 'req-5f1caa4c-562c-43b4-836f-4788c57fcd0e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.561475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] The token cache is being invalidate because project 445d73ace83e4fc8be9bb229fccce8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:18.561777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.562412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.564664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 987/1974] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:18 2022] DELETE /identity/v3/projects/445d73ace83e4fc8be9bb229fccce8cd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:18.566571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] The token cache is being invalidate because project cf4f846304ca417597295c3077cfe197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:18.567030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.567750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f1caa4c-562c-43b4-836f-4788c57fcd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.568544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.569450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 988/1975] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:18 2022] DELETE /identity/v3/projects/cf4f846304ca417597295c3077cfe197 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:18.569919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.575220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371e5fe-06cc-44ba-98b6-488a99a708e8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.575688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371e5fe-06cc-44ba-98b6-488a99a708e8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.575778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371e5fe-06cc-44ba-98b6-488a99a708e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.600481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e371e5fe-06cc-44ba-98b6-488a99a708e8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:18.601603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:18.601728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e371e5fe-06cc-44ba-98b6-488a99a708e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.651045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9333f0-6cd6-417c-96f2-cdd8d4d45929 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.667117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 989/1976] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:38:18.675292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.675614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.675925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.697069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:18.698274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.707287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 990/1977] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:38:18.710778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.711395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.711551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.711700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.711959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] PATH_INFO: `/v3/users/d6e5809875b74eeb9467eb1aafecefbe` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.713099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e5809875b74eeb9467eb1aafecefbe)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.715072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.722255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6c3e-5624-49d7-ac2b-eaa2b669223e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.726993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.727594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.727875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.728155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.728799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:18.729108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.730745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.744720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 991/1978] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:38:18.750283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.760232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d0bb9-4ac3-4f81-a7b9-639b7cdec58c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.764941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.765536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.765818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.766106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.767001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.768995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.771092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e5809875b74eeb9467eb1aafecefbe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.771752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e5809875b74eeb9467eb1aafecefbe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.772460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e5809875b74eeb9467eb1aafecefbe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.779750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e5809875b74eeb9467eb1aafecefbe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.794169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e5809875b74eeb9467eb1aafecefbe'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:18.803296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 992/1979] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:18 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:38:18.804319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.805969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:18.810635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 988/1980] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:18 2022] DELETE /identity/v3/users/d6e5809875b74eeb9467eb1aafecefbe => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:18.867621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:18.880762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdca1e2-74fd-44d1-aeec-c84715529912 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.945001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:18.945618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:18.945927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:18.946204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:18.946880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:18.949108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:18.981403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 993/1981] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:18 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:18.982756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:18.995055 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f12354-e51e-4001-a20a-4d8d97c32c98 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:18.999692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.000357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.000645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.001845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.002838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.004762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.018118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 989/1982] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:19.021744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.032181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197133-5c53-4c02-9101-2e2490cfba42 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.037505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.038156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.038503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.038777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.039507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.041246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.054013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 994/1983] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:19.058074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.079936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1625a-c17c-42fa-aedb-ad3d9a3e423a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.085260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.086226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.086681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.087085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] PATH_INFO: `/v3/projects/65fd8c340a694ecb9043eeab659dbe23/users/16c7bbe85a254903855bd1cc1930f49b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.105117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fd8c340a694ecb9043eeab659dbe23, user_id=16c7bbe85a254903855bd1cc1930f49b, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.107885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2457e7-0c0c-40fb-b975-b6f28719bfea admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.130391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 990/1984] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:19 2022] PUT /identity/v3/projects/65fd8c340a694ecb9043eeab659dbe23/users/16c7bbe85a254903855bd1cc1930f49b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:19.131698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.143462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.143672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.143979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.144261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.159297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f4f4d-6acc-4242-b05c-a34fade423ed admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.176193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] MFA Rules not processed for user `16c7bbe85a254903855bd1cc1930f49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:19.177434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.181750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:19.222028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.222681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.222964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.223234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] PATH_INFO: `/v3/projects/f0c330357d8c4e9098e7644c3688005d` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.224411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c330357d8c4e9098e7644c3688005d)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.227504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.252408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 991/1985] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:19.264453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.265825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.282460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c330357d8c4e9098e7644c3688005d', 'request_id': 'req-a532bef0-f4b6-4927-99bc-41a9e5af4740'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:19.288931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c330357d8c4e9098e7644c3688005d', 'request_id': 'req-a532bef0-f4b6-4927-99bc-41a9e5af4740'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:19.298457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c330357d8c4e9098e7644c3688005d', 'request_id': 'req-a532bef0-f4b6-4927-99bc-41a9e5af4740'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:19.332613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] The token cache is being invalidate because project f0c330357d8c4e9098e7644c3688005d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:19.332996 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:19.335637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a532bef0-f4b6-4927-99bc-41a9e5af4740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:19.340857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 995/1986] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:19 2022] DELETE /identity/v3/projects/f0c330357d8c4e9098e7644c3688005d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:19.348814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88a77-218a-4df4-bb07-d300cb1d3410 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.348951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88a77-218a-4df4-bb07-d300cb1d3410 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.349354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d88a77-218a-4df4-bb07-d300cb1d3410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.350150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a168-25e9-432c-ba85-42eff55c1d48 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.378239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b8d88a77-218a-4df4-bb07-d300cb1d3410 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:19.380514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b8d88a77-218a-4df4-bb07-d300cb1d3410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.390468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:19.419837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.420878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.421407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.421661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.424353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.476351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 996/1987] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:19.488251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.488580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.488884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.511370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.513207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.514572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:19.515822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.525441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 997/1988] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:38:19.530234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.537560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 992/1989] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:19.540675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410a894-1d8a-4b0f-8c3e-ebd25f5bb64b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.545321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.545924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.546206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.546480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.547204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:19.547511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.549961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.561889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 998/1990] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:38:19.566544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.577681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8c8b1-fc9b-45ec-ad22-f1d8f8078d35 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.582508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.583173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.583579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.583836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.584485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.586353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.602404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:19.619737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 993/1991] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:19.621287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.632794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea5693-b768-4eb4-a240-26065625e045 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.694452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.695220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.695501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.695806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.696591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.699019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.725947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 999/1992] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:38:19.727134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.738558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f7d3-8255-4d5a-8151-ca69ee50f86b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.743493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.744106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.745012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.745012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.745309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.747017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.759376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 994/1993] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:19.763436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.774589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660084c-7313-4c5f-b0ad-b005eb8c2920 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.779434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.780062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.780343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.780622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.781266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.784472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d96cf5-a118-4556-a2a6-aebede8cfa93 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.796832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1000/1994] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:19.804270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:19.815858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3647-93f1-4a97-a15f-2fb036e57847 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:19.821060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:19.821808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.822164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.822524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] PATH_INFO: `/v3/projects/d709fe9dff924ec081f3cb79aef01694/users/a7c50abef861423d99cfa0080adc82d9/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.839831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d709fe9dff924ec081f3cb79aef01694, user_id=a7c50abef861423d99cfa0080adc82d9, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:19.842557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:19.861437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 995/1995] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:19 2022] PUT /identity/v3/projects/d709fe9dff924ec081f3cb79aef01694/users/a7c50abef861423d99cfa0080adc82d9/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:19.872483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:19.873034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:19.873493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:19.905509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] MFA Rules not processed for user `a7c50abef861423d99cfa0080adc82d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:19.906760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:19.968164 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:19.975598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1001/1996] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:19.980108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.050611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ed0f-d37e-4ad1-ac25-3c7e7bbe83dc admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.055139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.055741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.056022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.056295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.056927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.058845 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.090515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 996/1997] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:19 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:20.095506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.106817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad0fa3-69c2-4673-a8b6-c009bb5d7dc3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.164157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.164772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.165062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.165338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.165966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.167944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.190978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1002/1998] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:20 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:20.195035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.205665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f60613-89a3-4dbd-afb0-56a37c1a5054 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.210403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.211047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.211332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.211610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.212300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.213902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.225903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 997/1999] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:20.229978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.241107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cde747-4466-4a5c-91c8-e5246e9abb0b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.246297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.247089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.247344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.247685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] PATH_INFO: `/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0/users/991ad55bf438438c8331cbd787fc319b/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.262664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbd7424395a4ee2bca12146a7d1e3a0, user_id=991ad55bf438438c8331cbd787fc319b, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.265336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.279291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1003/2000] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:20 2022] PUT /identity/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0/users/991ad55bf438438c8331cbd787fc319b/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:20.283851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.293851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7431282-a158-4793-9c09-a8b107ddfba4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.359027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.359716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.359997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.360333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.360917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.362710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20590db-9040-46d1-9698-419a719d0852 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.373359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 998/2001] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:38:20.378209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.389000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bba72-f125-4f88-8f43-c8f26f82eff2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.394599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.395321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.395604 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.395878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] PATH_INFO: `/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0/users/991ad55bf438438c8331cbd787fc319b/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.397850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbd7424395a4ee2bca12146a7d1e3a0, user_id=991ad55bf438438c8331cbd787fc319b, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.400626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.411061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1004/2002] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:20 2022] PUT /identity/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0/users/991ad55bf438438c8331cbd787fc319b/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:20.421328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.421621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.421902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.453474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] MFA Rules not processed for user `991ad55bf438438c8331cbd787fc319b`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:20.455700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:20.474325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:20.543983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 999/2003] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:20.552851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.555738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:20.630054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b85083-2fb2-4c26-9d7d-52b194e19621 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.634173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.634884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.635232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.635568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.646605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.648787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece891a2-5f52-4d9a-a793-082a9aedabc3 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.663334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1005/2004] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:20.766813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:20.835231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ead88-aff6-4b15-afd9-3a21496471d3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:20.839899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:20.840519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.840796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.841067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] PATH_INFO: `/v3/users/a7c50abef861423d99cfa0080adc82d9` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.842179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c50abef861423d99cfa0080adc82d9)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:20.844319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:20.917688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c50abef861423d99cfa0080adc82d9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:20.918387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c50abef861423d99cfa0080adc82d9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:20.919022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c50abef861423d99cfa0080adc82d9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:20.929744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c50abef861423d99cfa0080adc82d9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:20.943287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c50abef861423d99cfa0080adc82d9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:20.956217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1000/2005] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:20 2022] DELETE /identity/v3/users/a7c50abef861423d99cfa0080adc82d9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:20.960793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:20.961087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:20.961373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:20.981862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:20.983109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:21.031480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:21.046255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1006/2006] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:21.056778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:21.058185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:21.093222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:21.118918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963805c7-0fb0-405b-9ed5-c9332ba199b4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:21.122480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:21.123112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:21.123389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:21.123658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:21.133217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:21.134592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:21.145674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1001/2007] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:21.440067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:21.441846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:21.456260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfd586-f025-4532-8d2a-ee0d70fb4305 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:21.461276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:21.462019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:21.462381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:21.462766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] PATH_INFO: `/v3/projects/d709fe9dff924ec081f3cb79aef01694` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:21.465050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d709fe9dff924ec081f3cb79aef01694)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:21.467512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:21.540780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd709fe9dff924ec081f3cb79aef01694', 'request_id': 'req-97016e84-cac4-4140-9610-b533c262bd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.546808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd709fe9dff924ec081f3cb79aef01694', 'request_id': 'req-97016e84-cac4-4140-9610-b533c262bd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.553280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd709fe9dff924ec081f3cb79aef01694', 'request_id': 'req-97016e84-cac4-4140-9610-b533c262bd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.578341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] The token cache is being invalidate because project d709fe9dff924ec081f3cb79aef01694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:21.579323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.580078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.589222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1007/2008] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:21 2022] DELETE /identity/v3/projects/d709fe9dff924ec081f3cb79aef01694 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:21.589574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:21.600569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753eda-2011-46da-b903-a89d26eed0a6 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:21.671683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:21.672899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:21.673297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:21.673626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] PATH_INFO: `/v3/users/991ad55bf438438c8331cbd787fc319b` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:21.675407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991ad55bf438438c8331cbd787fc319b)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:21.677891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:21.732552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991ad55bf438438c8331cbd787fc319b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.733678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991ad55bf438438c8331cbd787fc319b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.734081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991ad55bf438438c8331cbd787fc319b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.742971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991ad55bf438438c8331cbd787fc319b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.758323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-723f86d0-ef20-460e-9342-ae71797d10a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991ad55bf438438c8331cbd787fc319b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:21.775301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1002/2009] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:21 2022] DELETE /identity/v3/users/991ad55bf438438c8331cbd787fc319b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:22.174661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.176556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:22.193224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97016e84-cac4-4140-9610-b533c262bd8c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.262028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.262698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.262987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.263264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] PATH_INFO: `/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.264430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbd7424395a4ee2bca12146a7d1e3a0)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.267021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.313151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbd7424395a4ee2bca12146a7d1e3a0', 'request_id': 'req-0eb54169-11cb-4ce3-853c-78c7b107d464'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:22.319861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbd7424395a4ee2bca12146a7d1e3a0', 'request_id': 'req-0eb54169-11cb-4ce3-853c-78c7b107d464'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:22.326255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbd7424395a4ee2bca12146a7d1e3a0', 'request_id': 'req-0eb54169-11cb-4ce3-853c-78c7b107d464'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:22.339794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] The token cache is being invalidate because project 0dbd7424395a4ee2bca12146a7d1e3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:22.340084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:22.340709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb54169-11cb-4ce3-853c-78c7b107d464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:22.342247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1008/2010] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:22 2022] DELETE /identity/v3/projects/0dbd7424395a4ee2bca12146a7d1e3a0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:22.347782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.348127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.348409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.368656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:22.369867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:22.418165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:22.431530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1003/2011] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:22.441171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.441474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.441752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.456542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:22.462082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:22.464064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:22.473473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1009/2012] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:22.477862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.488288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04a88-8f90-484d-b9fa-64f262ba8ea2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.492903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.493507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.493787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.494060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.494743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:22.495049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.496645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.508145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1004/2013] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:22.513108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.522972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bd180-e5e2-4585-a417-347698a23956 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.527690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.528286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.528598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.528874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.529499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.531498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.564502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1010/2014] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:22.568737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.579276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff0566-b7a1-4b3d-8cfc-01654b13b542 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.639042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.639749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.640101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.640383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.641025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.643731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.667384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1005/2015] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:22.671724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.683503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4d75-d635-4569-a8c7-9b524bc02432 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.689157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.689811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.690123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.690398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.691082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.693153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.708370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1011/2016] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:22.710010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.725338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6cd2-556c-4f57-ae3a-da5da1077489 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.730299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.731000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.731289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.731704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.732370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.734121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5be8d-c512-4b31-ac43-9082bbf31dfb admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.746589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1006/2017] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:22.751304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:22.761537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c08ea7-db8f-476c-a19c-8a23f738401e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:22.766222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:22.767474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.767754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.768085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] PATH_INFO: `/v3/projects/631668ae344447509fcc788c6ce535ec/users/ab18355823df4e77b90bc84b8ded5bd5/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.785686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631668ae344447509fcc788c6ce535ec, user_id=ab18355823df4e77b90bc84b8ded5bd5, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:22.788693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:22.806076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1012/2018] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:22 2022] PUT /identity/v3/projects/631668ae344447509fcc788c6ce535ec/users/ab18355823df4e77b90bc84b8ded5bd5/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:22.818463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:22.818884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:22.819254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:22.852121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] MFA Rules not processed for user `ab18355823df4e77b90bc84b8ded5bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:22.853542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:22.944970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1007/2019] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:22.946132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.048700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0778ef-d088-44e7-9b6f-c5e6d30dfac1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.057214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.058136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.065646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.065646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.065646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.065646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.100668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1013/2020] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:22 2022] POST /identity/v3/projects => generated 409 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:23.105684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.124531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c674742-a178-4127-8bc8-a48ae996c032 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.191987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.192605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.192905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.193166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.193795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.198548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.201321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:23.229311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1008/2021] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:23 2022] POST /identity/v3/users => generated 311 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:23.230499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.242250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b813-ed91-4cf0-bbdf-5a84b2338109 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.258890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.258890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.258890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.258890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.259425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.259425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.263264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1014/2022] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:23.267429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.277440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be95bf0-b52d-4b55-b08b-85572749b3ca admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.282664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.283325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.283648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.283887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] PATH_INFO: `/v3/projects/25a2bb4e9ea64acd90623520927f3c57/users/f7bddb889bca45e1b2efa800add69bd8/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.300565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a2bb4e9ea64acd90623520927f3c57, user_id=f7bddb889bca45e1b2efa800add69bd8, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.303405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.321676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1009/2023] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:23 2022] PUT /identity/v3/projects/25a2bb4e9ea64acd90623520927f3c57/users/f7bddb889bca45e1b2efa800add69bd8/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:23.325800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.336097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a572ef1-483c-4d16-b7ba-702dd8ea4d92 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.392978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.393636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.393991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.394301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.395111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.396887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af48d09-1962-4396-80eb-f0034ffeb1cf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.407473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1015/2024] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:23.411825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.422217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be82159-cdab-446c-9131-6ad0f0453b5c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.427719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.428373 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.428655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.428927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] PATH_INFO: `/v3/projects/25a2bb4e9ea64acd90623520927f3c57/users/f7bddb889bca45e1b2efa800add69bd8/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.430977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a2bb4e9ea64acd90623520927f3c57, user_id=f7bddb889bca45e1b2efa800add69bd8, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.433736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.446309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1010/2025] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:23 2022] PUT /identity/v3/projects/25a2bb4e9ea64acd90623520927f3c57/users/f7bddb889bca45e1b2efa800add69bd8/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:23.457827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.458116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.458394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.484470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:23.488072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] MFA Rules not processed for user `f7bddb889bca45e1b2efa800add69bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:23.490056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:23.563695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1016/2026] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:23 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:23.574705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.576459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:23.639420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6da7b0-6e53-452e-8b82-a3068a47a18f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.643993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.644613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.644900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.645181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.654776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.656215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.666817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1011/2027] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:23 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:38:23.820149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:23.830046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d667c1-9881-45a5-91a6-89e5ab7996bb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:23.833761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:23.834361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:23.834657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:23.834943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:23.892815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:23.894532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c9a71-4f9e-4598-b487-233c6360256f service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:23.905093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1017/2028] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:23 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:33.662076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:33.903311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=56 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:46.981317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:47.042694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d8284-64c5-41fd-968c-41eac25bd64c service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:47.048284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:47.049246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:47.049246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:47.049512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] PATH_INFO: `/v3/users/ab18355823df4e77b90bc84b8ded5bd5` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:47.050660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab18355823df4e77b90bc84b8ded5bd5)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:47.052628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:47.101971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab18355823df4e77b90bc84b8ded5bd5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.102593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab18355823df4e77b90bc84b8ded5bd5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.103442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab18355823df4e77b90bc84b8ded5bd5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.111869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab18355823df4e77b90bc84b8ded5bd5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.123709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab18355823df4e77b90bc84b8ded5bd5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.136524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1012/2029] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:46 2022] DELETE /identity/v3/users/ab18355823df4e77b90bc84b8ded5bd5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:47.141177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:47.141657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:47.141803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:47.160507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:47.161701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:47.220375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1018/2030] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:47.230389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:47.231927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:47.291630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350956-9d38-4141-83a4-27d6b9bd42ce admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:47.295391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:47.296001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:47.296278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:47.296690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:47.307567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:47.308662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:47.319599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1013/2031] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:47.851745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:47.853154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:47.865645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931615-a04d-447a-a433-4b251c4e2906 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:47.870169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:47.870823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:47.871105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:47.871378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] PATH_INFO: `/v3/users/0e295e00b2844af282b31873a8612871` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:47.872494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e295e00b2844af282b31873a8612871)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:47.874425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:47.932400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e295e00b2844af282b31873a8612871'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.933074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e295e00b2844af282b31873a8612871'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.941129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e295e00b2844af282b31873a8612871'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.941961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e295e00b2844af282b31873a8612871'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.950602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7a9a976f-6e80-4bb0-a284-b7d27858e33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e295e00b2844af282b31873a8612871'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:47.956011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:47.958582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:47.970534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1019/2032] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:47 2022] DELETE /identity/v3/users/0e295e00b2844af282b31873a8612871 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:47.974019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6034cf9-637e-49cc-a3dc-bca6fe4312a9 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:47.975342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:47.975626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:47.976466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:47.997076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:47.997608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:47.998723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.048209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:48.048851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.049358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.049640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] PATH_INFO: `/v3/projects/631668ae344447509fcc788c6ce535ec` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.050815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631668ae344447509fcc788c6ce535ec)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:48.052970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:48.071385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1020/2033] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:38:48.083305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:48.084808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.103622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:48.107820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631668ae344447509fcc788c6ce535ec', 'request_id': 'req-448cce82-6ffc-465a-a789-7137bb572525'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.116768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631668ae344447509fcc788c6ce535ec', 'request_id': 'req-448cce82-6ffc-465a-a789-7137bb572525'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.126044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631668ae344447509fcc788c6ce535ec', 'request_id': 'req-448cce82-6ffc-465a-a789-7137bb572525'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.145149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] The token cache is being invalidate because project 631668ae344447509fcc788c6ce535ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:48.145503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.146260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.148169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1014/2034] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:47 2022] DELETE /identity/v3/projects/631668ae344447509fcc788c6ce535ec => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:48.152263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:48.153862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.194478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f14505-8e39-4cf4-8dab-5e5d04bced08 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:48.234161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cce82-6ffc-465a-a789-7137bb572525 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:48.239860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:48.240755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.241188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.241611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] PATH_INFO: `/v3/users/f7bddb889bca45e1b2efa800add69bd8` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.243261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bddb889bca45e1b2efa800add69bd8)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:48.245267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:48.279559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:48.280171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.280451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.280782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.283556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.299960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:48.301421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:48.316395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bddb889bca45e1b2efa800add69bd8'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.317323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bddb889bca45e1b2efa800add69bd8'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.318095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bddb889bca45e1b2efa800add69bd8'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.326309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1021/2035] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:48.332624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bddb889bca45e1b2efa800add69bd8'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.334562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:48.353109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-582e182c-6bab-4ead-8ab8-b576979df8df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bddb889bca45e1b2efa800add69bd8'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.368341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1015/2036] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:48 2022] DELETE /identity/v3/users/f7bddb889bca45e1b2efa800add69bd8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:48.526039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:48.529107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.549607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8fed-57d9-428a-a2b0-3e70f4cd5cc2 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:48.637066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:48.637808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.638161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.639058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] PATH_INFO: `/v3/users/16c7bbe85a254903855bd1cc1930f49b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.639923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c7bbe85a254903855bd1cc1930f49b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:48.642423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:48.709214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c7bbe85a254903855bd1cc1930f49b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.709898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c7bbe85a254903855bd1cc1930f49b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.718301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c7bbe85a254903855bd1cc1930f49b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.719676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c7bbe85a254903855bd1cc1930f49b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.728980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c7bbe85a254903855bd1cc1930f49b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:48.739395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:48.761294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.761602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.761865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.762726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1022/2037] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:48 2022] DELETE /identity/v3/users/16c7bbe85a254903855bd1cc1930f49b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:48.807224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:48.810062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.856810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:48.863273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.922806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1016/2038] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:48.923694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:48.925131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:48.951348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec007e-dd7d-46ca-bb00-dfba989797a3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:48.963156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:48.964384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:48.964910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:48.965480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] PATH_INFO: `/v3/projects/25a2bb4e9ea64acd90623520927f3c57` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:48.967171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a2bb4e9ea64acd90623520927f3c57)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:48.969914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.020648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc118f74-26bb-475b-a023-bdbe20115f03 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.027957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.028834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.029117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.029492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] PATH_INFO: `/v3/projects/a167db5e9c164070a21feb916e1906cb` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.034947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a167db5e9c164070a21feb916e1906cb)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.039522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.098046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a2bb4e9ea64acd90623520927f3c57', 'request_id': 'req-ef745361-0a10-465f-96ab-4cd404aa27e2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.105848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a2bb4e9ea64acd90623520927f3c57', 'request_id': 'req-ef745361-0a10-465f-96ab-4cd404aa27e2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.111618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a167db5e9c164070a21feb916e1906cb', 'request_id': 'req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.114844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a2bb4e9ea64acd90623520927f3c57', 'request_id': 'req-ef745361-0a10-465f-96ab-4cd404aa27e2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.125101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a167db5e9c164070a21feb916e1906cb', 'request_id': 'req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.135954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a167db5e9c164070a21feb916e1906cb', 'request_id': 'req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.138721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] The token cache is being invalidate because project 25a2bb4e9ea64acd90623520927f3c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:49.139031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.144461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.146505 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1017/2039] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:48 2022] DELETE /identity/v3/projects/25a2bb4e9ea64acd90623520927f3c57 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:49.150314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.155690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:49.158536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] The token cache is being invalidate because project a167db5e9c164070a21feb916e1906cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:49.158949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.159712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-affeb2f5-7144-4035-b2f7-1a06bdbf62c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.162418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1023/2040] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:48 2022] DELETE /identity/v3/projects/a167db5e9c164070a21feb916e1906cb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:49.163831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.164144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.164490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.188681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:49.190049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:49.273061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef745361-0a10-465f-96ab-4cd404aa27e2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.279293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1024/2041] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:49.281574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.282871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:49.284501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:49.305723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f825c70-57d7-444e-9339-f55f2488f8da None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.329889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.331320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.332646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.333076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] PATH_INFO: `/v3/users/0d4fadbfd2ee4261b52bf2998b9cdd08` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.335039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4fadbfd2ee4261b52bf2998b9cdd08)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.339078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.363761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:49.365871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.366610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.367134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.367509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.382737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:49.406855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.409253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350327e-cde7-4245-89f7-d4710f57d694 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.419108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4fadbfd2ee4261b52bf2998b9cdd08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.420143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4fadbfd2ee4261b52bf2998b9cdd08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.422731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1018/2042] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:49.424102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.424389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.424670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.431197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4fadbfd2ee4261b52bf2998b9cdd08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.432163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4fadbfd2ee4261b52bf2998b9cdd08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.448290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4fadbfd2ee4261b52bf2998b9cdd08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:49.451207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:49.452589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:49.468644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1019/2043] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:49.470460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.488797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1025/2044] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:49 2022] DELETE /identity/v3/users/0d4fadbfd2ee4261b52bf2998b9cdd08 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:49.489085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad9db-93b9-4acf-8a38-2228a24cf4c3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.567950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.568549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.568832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.569111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.569766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:49.572520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.574324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.593001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1020/2045] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:49.599214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.618034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55b45d-7dc7-4b8f-a022-be3aa1b3eed0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.624386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.625960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.626331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.626871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.627420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.630426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.679742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1026/2046] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:49 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:49.683465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.701153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373f32-12ac-420a-9834-7b131550eee5 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.809607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.810247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.810757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.811851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.812148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.815705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.878572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.879889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1021/2047] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:49 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:49.891908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddd2c-66a8-4e3b-92e5-6a9d81ef93a6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.902115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.903429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.904151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.904725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.906806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.909972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:49.927161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1027/2048] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:49.931803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:49.954570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5f1a-98c8-45e8-b058-d4757f8b7f4c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:49.974340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:49.975666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:49.978890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:49.979881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:49.981441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:49.995582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.031597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1022/2049] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:50.034417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.037738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.037979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.059673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bed0d-39f3-4f99-af34-0e6cc1f34d2c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.068402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4029-4f5f-4f90-956a-42fe715ac2a8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.074757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.075539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.076388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.076388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] PATH_INFO: `/v3/projects/c5371b99224a4a3e92f46cb440945692/users/dccc7b85bd724eeba0d16fcc0562c02a/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.080723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.081356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.081647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.081969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] PATH_INFO: `/v3/projects/65fd8c340a694ecb9043eeab659dbe23` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.086232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fd8c340a694ecb9043eeab659dbe23)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.088219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.104780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5371b99224a4a3e92f46cb440945692, user_id=dccc7b85bd724eeba0d16fcc0562c02a, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.107844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.112420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:50.135459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1023/2050] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:50 2022] PUT /identity/v3/projects/c5371b99224a4a3e92f46cb440945692/users/dccc7b85bd724eeba0d16fcc0562c02a/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:50.145908 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.150987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.194738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fd8c340a694ecb9043eeab659dbe23', 'request_id': 'req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.205217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fd8c340a694ecb9043eeab659dbe23', 'request_id': 'req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.213300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fd8c340a694ecb9043eeab659dbe23', 'request_id': 'req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.236688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] The token cache is being invalidate because project 65fd8c340a694ecb9043eeab659dbe23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:50.237007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.239662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a2bf0704-dacb-48fa-a2e9-9c53ad75b77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.254679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1028/2051] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:50 2022] DELETE /identity/v3/projects/65fd8c340a694ecb9043eeab659dbe23 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:50.254679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.254679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.254679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.342459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] MFA Rules not processed for user `dccc7b85bd724eeba0d16fcc0562c02a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:50.351099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.370183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d956446-69c5-4ff0-8513-ef0fc8bca39e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.404550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:50.490474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1029/2052] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:50.495672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.497259 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.503499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.504190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.505168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.505168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] PATH_INFO: `/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.510100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a92494eaa8e4edab4d556b5b9d1c98b)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.513772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.517233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff215b-e6f7-480a-936f-ea3582b152db None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.524134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.524846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.525181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.525530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.526249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.528680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.586095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1030/2053] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:50.589676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.606969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a92494eaa8e4edab4d556b5b9d1c98b', 'request_id': 'req-1e67d0af-3e33-4a65-a712-a032aeefbad7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.615611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a92494eaa8e4edab4d556b5b9d1c98b', 'request_id': 'req-1e67d0af-3e33-4a65-a712-a032aeefbad7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.623105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a92494eaa8e4edab4d556b5b9d1c98b', 'request_id': 'req-1e67d0af-3e33-4a65-a712-a032aeefbad7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.638787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] The token cache is being invalidate because project 4a92494eaa8e4edab4d556b5b9d1c98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:50.639080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.639930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1e67d0af-3e33-4a65-a712-a032aeefbad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:50.643149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1024/2054] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:50 2022] DELETE /identity/v3/projects/4a92494eaa8e4edab4d556b5b9d1c98b => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:50.648168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94ac38-b79b-4714-8c0d-780bd21daba3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.648562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94ac38-b79b-4714-8c0d-780bd21daba3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.648987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94ac38-b79b-4714-8c0d-780bd21daba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.665947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ecea-8e2d-486d-ab7e-1ed6d4269dbf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.671311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7d94ac38-b79b-4714-8c0d-780bd21daba3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:50.673841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d94ac38-b79b-4714-8c0d-780bd21daba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.709077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:50.760433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.761688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.762439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.766674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.767655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.769830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.775454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1025/2055] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:50.783347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.783727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.784063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.803657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1031/2056] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:38:50.807133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.808825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.823025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:50.824243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:50.829138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2eed2-63b8-474d-81c7-01753bda5c91 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.833398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1026/2057] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:50.838868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.839270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.840290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.840781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.841167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.841953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.844486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.852240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e2f99-2a5f-4c3b-872e-6c976120c500 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.858465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.859896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.860412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.861130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.863195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:50.863735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.865583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.871200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1032/2058] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:38:50.875132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.882123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1027/2059] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:50.887008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.888834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f6211-c10a-4394-9050-6eef73226b99 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.897376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.898036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.898322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.898601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.899315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.900378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbefe67-bb88-490e-b851-f554da74519a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.901686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.909354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.910120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.910411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.910732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.911394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.913571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.914230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1033/2060] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:50.918803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.930997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc248cc-cd9e-4ad5-ab6f-ec895001be97 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:50.937175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:50.938195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.938524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.938836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] PATH_INFO: `/v3/projects/ad76ddacc9f24f3887f3a46acc1f7323/users/367de0a436c746178981fefa1b5d6649/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:50.953618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1028/2061] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/projects => generated 361 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:50.958272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:50.961070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad76ddacc9f24f3887f3a46acc1f7323, user_id=367de0a436c746178981fefa1b5d6649, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:50.964666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48798beb-6726-442d-a9c8-ededd412f87e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:50.994814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1034/2062] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:50 2022] PUT /identity/v3/projects/ad76ddacc9f24f3887f3a46acc1f7323/users/367de0a436c746178981fefa1b5d6649/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:50.996431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:50.996840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:50.997064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.006578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:51.038308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] MFA Rules not processed for user `367de0a436c746178981fefa1b5d6649`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:51.039691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:51.049712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e1662-d11b-46bb-876b-7c4491103198 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.163220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.165357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.165357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.165357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.165807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.168730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.174844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1035/2063] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:51.180152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.198105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cc64b-edf8-4fea-88d7-20cfeceb2b49 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.204401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1029/2064] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:50 2022] POST /identity/v3/users => generated 281 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:38:51.207864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.208756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.209530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.210042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.210439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.211256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.213288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.233780 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394995d-5f87-4aa6-a89e-10738ec7ad76 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.243201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.243851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.244274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.244475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.245332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.250532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.262965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1036/2065] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:51 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:51.268999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.273130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1030/2066] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:51.280108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.296408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b42cb2-188c-495c-9305-36724fcaa0c4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.334006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:51.384264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0d03-5e5e-4970-be59-cce2fcbcf087 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.392194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.393471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.393898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.394274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.395031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.398432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.401199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.403512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.403819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.404157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.405169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.407619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.426339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1031/2067] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:51.440268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1037/2068] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:51 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:51.442064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.448490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.467627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e71dfe-a859-4340-abae-183e66353a0d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.476539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9429f8-f322-43ff-8aa8-0c518b5840e0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.488731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.489524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.490058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.490473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.491332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.493114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.494069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.496328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.496623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.496903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] PATH_INFO: `/v3/projects/830ea75638a64082b8ff0d24ec8181a4/users/f8d474036e604085b9c28446096b3b56/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.512643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1038/2069] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:51.513767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.527261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d851e-4c9c-4431-b13d-f3ad977bc14b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.533246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.534418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.534677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.535137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] PATH_INFO: `/v3/projects/b4e3a7548deb43948c3aa2b573d626db/users/de735818b1c94f5eb3e4efde2f0d49d1/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.553321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ea75638a64082b8ff0d24ec8181a4, user_id=f8d474036e604085b9c28446096b3b56, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.556157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef7f9a-3882-4caa-842a-cc1f5375de14 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.559061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e3a7548deb43948c3aa2b573d626db, user_id=de735818b1c94f5eb3e4efde2f0d49d1, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.562220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.598685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1032/2070] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:51 2022] PUT /identity/v3/projects/830ea75638a64082b8ff0d24ec8181a4/users/f8d474036e604085b9c28446096b3b56/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:51.598924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1039/2071] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:51 2022] PUT /identity/v3/projects/b4e3a7548deb43948c3aa2b573d626db/users/de735818b1c94f5eb3e4efde2f0d49d1/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:51.603207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40128f-8507-4853-b44d-8870fe0ff192 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.603769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40128f-8507-4853-b44d-8870fe0ff192 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.604238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40128f-8507-4853-b44d-8870fe0ff192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.604831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.660781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6d40128f-8507-4853-b44d-8870fe0ff192 None None] MFA Rules not processed for user `f8d474036e604085b9c28446096b3b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:51.662206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6d40128f-8507-4853-b44d-8870fe0ff192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:51.670241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:51.740460 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b817-24c6-4db9-a200-04f9120b5fc5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.765467 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.767449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.767739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.768019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.769769 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.775924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.802673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1040/2072] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:51.809187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.841457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b19-fa32-4bd5-bece-6a9da2df32ba admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:51.855416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:51.857382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1033/2073] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:51.858137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.858503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.858823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] PATH_INFO: `/v3/projects/b4e3a7548deb43948c3aa2b573d626db/users/de735818b1c94f5eb3e4efde2f0d49d1/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.863811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e3a7548deb43948c3aa2b573d626db, user_id=de735818b1c94f5eb3e4efde2f0d49d1, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:51.866754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:51.882762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1041/2074] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:51 2022] PUT /identity/v3/projects/b4e3a7548deb43948c3aa2b573d626db/users/de735818b1c94f5eb3e4efde2f0d49d1/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:51.897886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:51.897886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:51.897886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:51.923694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:51.927351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:51.984768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] MFA Rules not processed for user `de735818b1c94f5eb3e4efde2f0d49d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:51.987252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.036990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdfdf1-9d27-4ed7-82cd-197da5165af0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.042273 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.043158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.043517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.043870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.154037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.155948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.176109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1042/2075] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:52.195818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.212506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1034/2076] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:52.220572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.232076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.263560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:52.284685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d0eda-7893-4f31-b533-5334285163d2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.295608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.301745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.302119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.303326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.376428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.377953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.402130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1035/2077] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:38:52.403127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.407924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.408367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7bbf-895d-4397-897c-15226444fee7 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.416230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.417190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.417812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.418527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] PATH_INFO: `/v3/users/f8d474036e604085b9c28446096b3b56` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.420406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d474036e604085b9c28446096b3b56)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.423962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.426889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:52.446232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a3a30-cc15-45a5-9a41-73d23eaa0362 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.457934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.458905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.459462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.460098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] PATH_INFO: `/v3/users/b490535729c64eaa8e89bf8eb8d56290` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.462118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b490535729c64eaa8e89bf8eb8d56290)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.464792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.513156 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d474036e604085b9c28446096b3b56'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.514351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d474036e604085b9c28446096b3b56'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.535596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d474036e604085b9c28446096b3b56'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.536547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d474036e604085b9c28446096b3b56'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.555712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b490535729c64eaa8e89bf8eb8d56290'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.556680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b490535729c64eaa8e89bf8eb8d56290'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.560149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d474036e604085b9c28446096b3b56'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.560286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b490535729c64eaa8e89bf8eb8d56290'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.583250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b490535729c64eaa8e89bf8eb8d56290'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.600265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1043/2078] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:52 2022] DELETE /identity/v3/users/f8d474036e604085b9c28446096b3b56 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:52.601352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.602892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.635565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-090453b3-f0eb-4617-b514-7827b7360dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b490535729c64eaa8e89bf8eb8d56290'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.663639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1036/2079] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:52 2022] DELETE /identity/v3/users/b490535729c64eaa8e89bf8eb8d56290 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:52.665118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.669867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.670186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.707130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:52.709666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.710036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1cd7b-73c4-48c5-b2c4-7b1446fe9326 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.803792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.804668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.804797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.804981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.806926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.840096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1037/2080] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:52.841396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.853026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.872950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8236e8-a318-469a-988e-625ffd156878 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:52.875001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:52.882875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:52.883740 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:52.884059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:52.884352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] PATH_INFO: `/v3/users/b8340a5a706b4eaa984c5758351632ac` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:52.885898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8340a5a706b4eaa984c5758351632ac)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.887954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.900665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:52.902213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:52.913442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1044/2081] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:52.914144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:52.915489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:52.979361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8340a5a706b4eaa984c5758351632ac'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.980189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8340a5a706b4eaa984c5758351632ac'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.981199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8340a5a706b4eaa984c5758351632ac'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.989846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8340a5a706b4eaa984c5758351632ac'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:52.995517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:53.003805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6891301-617b-41ad-a1d5-bc56daef9315 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.006988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8340a5a706b4eaa984c5758351632ac'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.009658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.010281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.010675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.010955 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.012857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.025925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1038/2082] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:52 2022] DELETE /identity/v3/users/b8340a5a706b4eaa984c5758351632ac => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:53.025925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.025925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.035484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.037058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b25aad-25e1-420d-a994-f90f0cc70718 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.050121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1045/2083] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:53.101019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54c676-96f9-4db6-ae82-af275f8ba3fc admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.108130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.108776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.109108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.109576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] PATH_INFO: `/v3/projects/ee9eb7c5b5f94cd78546a3c1cc3d7cc7` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.111303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9eb7c5b5f94cd78546a3c1cc3d7cc7)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.113315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.191608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9eb7c5b5f94cd78546a3c1cc3d7cc7', 'request_id': 'req-0851f7fa-1080-4182-97f2-55a0eaab5ce7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.201511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9eb7c5b5f94cd78546a3c1cc3d7cc7', 'request_id': 'req-0851f7fa-1080-4182-97f2-55a0eaab5ce7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.211356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9eb7c5b5f94cd78546a3c1cc3d7cc7', 'request_id': 'req-0851f7fa-1080-4182-97f2-55a0eaab5ce7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.233539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] The token cache is being invalidate because project ee9eb7c5b5f94cd78546a3c1cc3d7cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:53.234252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.235655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.237454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1039/2084] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:53 2022] DELETE /identity/v3/projects/ee9eb7c5b5f94cd78546a3c1cc3d7cc7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:53.243092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20bad-b3f6-43f8-a954-51794f5a10b1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.243390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20bad-b3f6-43f8-a954-51794f5a10b1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.244198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20bad-b3f6-43f8-a954-51794f5a10b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.282376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.283959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.295496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-90d20bad-b3f6-43f8-a954-51794f5a10b1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:53.296783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-90d20bad-b3f6-43f8-a954-51794f5a10b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.378209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851f7fa-1080-4182-97f2-55a0eaab5ce7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.383134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.384412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.384706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.385121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] PATH_INFO: `/v3/projects/80c17e53479e4f88b363de447a5f932b` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.386736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c17e53479e4f88b363de447a5f932b)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.389181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.400418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:53.412487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1046/2085] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:53.422581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.423016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.424608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.492610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:53.494417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.499101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c17e53479e4f88b363de447a5f932b', 'request_id': 'req-55303b3f-15eb-4716-9f4c-661abe1b2906'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.505767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c17e53479e4f88b363de447a5f932b', 'request_id': 'req-55303b3f-15eb-4716-9f4c-661abe1b2906'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.513429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c17e53479e4f88b363de447a5f932b', 'request_id': 'req-55303b3f-15eb-4716-9f4c-661abe1b2906'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.528367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] The token cache is being invalidate because project 80c17e53479e4f88b363de447a5f932b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:53.528668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.529328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.530948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1040/2086] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:53 2022] DELETE /identity/v3/projects/80c17e53479e4f88b363de447a5f932b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:53.535086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.535086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.582411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1047/2087] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:53.583306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.584620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.596835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d87d85-fffb-4cdd-bcee-505506cb1841 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.601944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.602323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.602673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.602973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] PATH_INFO: `/v3/users/6ad19af637bb400eafd18a6709ab350e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.604126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad19af637bb400eafd18a6709ab350e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.606054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.614080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303b3f-15eb-4716-9f4c-661abe1b2906 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.620788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.622016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.622305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.622585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] PATH_INFO: `/v3/projects/830ea75638a64082b8ff0d24ec8181a4` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.623753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=830ea75638a64082b8ff0d24ec8181a4)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.626364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.642358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:53.657649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad19af637bb400eafd18a6709ab350e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.658423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad19af637bb400eafd18a6709ab350e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.665783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad19af637bb400eafd18a6709ab350e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.666415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad19af637bb400eafd18a6709ab350e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.674650 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad19af637bb400eafd18a6709ab350e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.677016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830ea75638a64082b8ff0d24ec8181a4', 'request_id': 'req-7f4d0241-8324-4f6b-9445-d5e526042c5e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.682944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830ea75638a64082b8ff0d24ec8181a4', 'request_id': 'req-7f4d0241-8324-4f6b-9445-d5e526042c5e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.689694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830ea75638a64082b8ff0d24ec8181a4', 'request_id': 'req-7f4d0241-8324-4f6b-9445-d5e526042c5e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.694094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1048/2088] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:53 2022] DELETE /identity/v3/users/6ad19af637bb400eafd18a6709ab350e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:53.695256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.696935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.705259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] The token cache is being invalidate because project 830ea75638a64082b8ff0d24ec8181a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:53.705608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.706275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f4d0241-8324-4f6b-9445-d5e526042c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:53.708411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1041/2089] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:53 2022] DELETE /identity/v3/projects/830ea75638a64082b8ff0d24ec8181a4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:53.712825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58194160-a1bf-4fc0-ac19-1f694abacf68 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.713112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58194160-a1bf-4fc0-ac19-1f694abacf68 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.713411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58194160-a1bf-4fc0-ac19-1f694abacf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.734038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-58194160-a1bf-4fc0-ac19-1f694abacf68 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:53.736195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-58194160-a1bf-4fc0-ac19-1f694abacf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.763365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:53.791448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af22b1-8cbb-48dd-bb7c-29cf07685e0f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.797628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.798492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.798612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.799054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.799584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:53.799942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.801498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.817618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1049/2090] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:53.818519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.826097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.826263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1042/2091] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:53.828620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.828911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.829186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.840584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dd3ff-8040-4b1c-95d1-6f0ea06619f7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.845577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.846875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.847161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.847437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.848068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.850074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:53.850202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.851408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:53.864502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1043/2092] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:53.865691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.878105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106488-1da4-4ab1-8879-405d23474fad None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.883508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:53.884303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:53.884580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:53.884877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:53.885589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:53.885902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:53.887578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:53.902508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1050/2093] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:53.903695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.906634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1044/2094] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:53.914194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:53.929239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075b86b-e390-42f3-b1f3-f9dd47a0d547 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:53.994183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc3789-62bb-45ee-b44f-d99d4b180a0f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.003036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.003657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.003945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.004248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.005011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.006886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.013080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.013699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.013985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.014566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.015119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.018480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.032359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1051/2095] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:54.033200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.034891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.047646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf99a2-cead-47e9-bcc0-3cfaed95c2e1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.053432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.054138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.054425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.054745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] PATH_INFO: `/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.055949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26de7a84fcd34fcb8b0c2e9bf5d1e948)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.057874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.060535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:54.065067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1045/2096] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:53 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:54.065067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.148211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26de7a84fcd34fcb8b0c2e9bf5d1e948', 'request_id': 'req-bf82145e-d65e-48ba-b058-c46ca4dbfc53'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:54.151838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b36f4-8cbb-4bf4-8a38-08d3ae5efa76 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.155252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26de7a84fcd34fcb8b0c2e9bf5d1e948', 'request_id': 'req-bf82145e-d65e-48ba-b058-c46ca4dbfc53'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:54.165224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26de7a84fcd34fcb8b0c2e9bf5d1e948', 'request_id': 'req-bf82145e-d65e-48ba-b058-c46ca4dbfc53'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:54.182194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] The token cache is being invalidate because project 26de7a84fcd34fcb8b0c2e9bf5d1e948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:54.182790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:54.184242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:54.191091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1052/2097] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:54 2022] DELETE /identity/v3/projects/26de7a84fcd34fcb8b0c2e9bf5d1e948 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:54.192017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.193446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.206264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:54.230145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.231436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.231895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.232336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.233153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.235134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bba32-40a0-49da-97c3-6299dc176eae admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.251165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1046/2098] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:54.253314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.253762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.254270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.264683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82145e-d65e-48ba-b058-c46ca4dbfc53 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.269912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.270960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.271406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.271832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.272610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.274823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc340458-8294-416e-84d2-de425b7f0d5e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.276573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:54.277811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.301564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1047/2099] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:54.307725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.307725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.317122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1053/2100] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:54.317122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.317122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.317122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.320514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c0363-a71a-4e67-b303-b3901f6c26df None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.325496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.326202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.326471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.326847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.327651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.330169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.344379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1048/2101] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:54.344379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.344740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:54.345936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.355201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1054/2102] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:54.356215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98718923-6360-45a5-a5ab-8200ee0a5848 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.359058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.362226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.362978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.363396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.363844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.364599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.366298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.369962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a0c87-62cc-49b0-9ea4-a0dc64b3ffeb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.374674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.375475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.375830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.376177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] PATH_INFO: `/v3/projects/947191f45d2749f6af46dc226da6ded6/users/82f70b90111f49ca859188da08044796/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.380804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1049/2103] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:54.380804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.389925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35f765-a2f2-4e3b-9b7e-66a3991ee6b2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.395652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.396332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.396520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=947191f45d2749f6af46dc226da6ded6, user_id=82f70b90111f49ca859188da08044796, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.396736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.397080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.397803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:54.398176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.399638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.399959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.414456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1050/2104] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:54.415807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.425268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1055/2105] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:54 2022] PUT /identity/v3/projects/947191f45d2749f6af46dc226da6ded6/users/82f70b90111f49ca859188da08044796/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:54.425268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.432855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e2582-34ca-475b-8ad8-175e4383ad0a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.487003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:54.505556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5dca-fdad-4aaf-b6fb-01dd491700e2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.509610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.510231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.510514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.510826 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.511133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.511484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.511814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.512095 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.512371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.513711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80695446-827d-4b88-b7e4-582bf5b9c2c3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.513841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.515797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.526578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1051/2106] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:54.527983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.528274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.528840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.559425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1056/2107] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:54.560685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.563623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] MFA Rules not processed for user `82f70b90111f49ca859188da08044796`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:54.565019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.645975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5242a53-5714-446f-9967-7065e054799a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.652264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.652984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.653276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.653571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] PATH_INFO: `/v3/projects/533cb508337149838cb6c2cf33a380a2/users/3b6c0f18db3446c8a0fa88848aa322f7/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.661526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1052/2108] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:54.661526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.672910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533cb508337149838cb6c2cf33a380a2, user_id=3b6c0f18db3446c8a0fa88848aa322f7, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.676195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.677170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f52bb-7b71-4c7e-879c-d90abda81a3a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.683218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.684477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.684985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.685375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.686175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.688902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bc3fd-d776-43d3-982f-ec3eed2a6d82 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.697084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1057/2109] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:54 2022] PUT /identity/v3/projects/533cb508337149838cb6c2cf33a380a2/users/3b6c0f18db3446c8a0fa88848aa322f7/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:54.698325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.700402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.732038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1053/2110] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/users => generated 288 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:54.733391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.733689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.733969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.741944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:54.774731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] MFA Rules not processed for user `3b6c0f18db3446c8a0fa88848aa322f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:54.776247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.803405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0437d-f49a-479b-a995-c9e69395e492 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.819652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.820504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.821109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.821388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.873988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1054/2111] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:54.875675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.887926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:54.891353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.895562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.908529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1058/2112] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:54.909621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.911794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:54.968129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5afb5-71b3-4e84-99b8-e94d3e4d2423 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.975002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.975651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.975937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.976206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.977055 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.979653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:54.981883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12881e31-5f25-4cee-ae87-0c31a0f9efcb service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:54.986209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:54.986853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:54.987136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:54.987408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:54.993575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1055/2113] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:54.995756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:54.996863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:54.998417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:55.008584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1059/2114] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:55.009113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bac33-db68-4a4f-b747-9cf3e7f5eabe admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:55.014124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:55.014702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:55.015007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:55.015266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:55.015986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:55.017632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ae50c-0212-4f30-9d79-8e12cf47cc67 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:55.029675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1056/2115] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:55.035682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:55.046193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaf194-53d5-4821-bb51-ee6c6baef25c service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:55.051193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:55.051847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:55.052132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:55.052559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] PATH_INFO: `/v3/projects/a043bf3e9d454ea0a576446ae005249f/users/bd10d229bed14abf9bf2ec85b7e4bac0/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:55.071735 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a043bf3e9d454ea0a576446ae005249f, user_id=bd10d229bed14abf9bf2ec85b7e4bac0, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:55.075205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:55.091980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1060/2116] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:55 2022] PUT /identity/v3/projects/a043bf3e9d454ea0a576446ae005249f/users/bd10d229bed14abf9bf2ec85b7e4bac0/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:55.103224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:55.103519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:55.103806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:55.135716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] MFA Rules not processed for user `bd10d229bed14abf9bf2ec85b7e4bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:55.137447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:55.232191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1057/2117] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:55.245998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:55.248502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:55.393061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa739fdc-b196-4f66-a304-b9033436dc1e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:55.402881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:55.403643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:55.404001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:55.404543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:55.434392 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:55.436282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613cfe5-26d8-4c7f-8d44-7f70ff0928f6 service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:55.475001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1061/2118] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:56.531504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:56.535206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:56.556483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:56.623609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d370b-7b61-40fe-8db9-6c17524ea64d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:56.629376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:56.630083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:56.630417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:56.630818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] PATH_INFO: `/v3/users/dccc7b85bd724eeba0d16fcc0562c02a` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:56.631961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccc7b85bd724eeba0d16fcc0562c02a)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:56.634170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:56.747962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccc7b85bd724eeba0d16fcc0562c02a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:56.749076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccc7b85bd724eeba0d16fcc0562c02a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:56.749727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccc7b85bd724eeba0d16fcc0562c02a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:56.761279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccc7b85bd724eeba0d16fcc0562c02a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:56.785706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccc7b85bd724eeba0d16fcc0562c02a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:56.803638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1058/2119] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:56 2022] DELETE /identity/v3/users/dccc7b85bd724eeba0d16fcc0562c02a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:56.808230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:56.808745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:56.808864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:56.828884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:56.830347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:56.916117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:56.928653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1062/2120] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:56.939350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:56.941001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:57.030669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7b9eb-b742-4bf3-9eb8-f35393cf74b2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:57.035356 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:57.035965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:57.036249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:57.036589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:57.052305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:57.054154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:57.068101 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1059/2121] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:57.561834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:57.563398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:57.575457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0bc7e-aade-4082-ac37-eb5c127b80ce None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:57.580884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:57.581455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:57.581736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:57.582009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] PATH_INFO: `/v3/projects/c5371b99224a4a3e92f46cb440945692` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:57.583214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5371b99224a4a3e92f46cb440945692)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:57.585139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:57.631346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5371b99224a4a3e92f46cb440945692', 'request_id': 'req-e4538692-2e3e-41fd-8829-7cf89e0f1333'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.636745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5371b99224a4a3e92f46cb440945692', 'request_id': 'req-e4538692-2e3e-41fd-8829-7cf89e0f1333'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.643437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5371b99224a4a3e92f46cb440945692', 'request_id': 'req-e4538692-2e3e-41fd-8829-7cf89e0f1333'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.656869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] The token cache is being invalidate because project c5371b99224a4a3e92f46cb440945692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:57.657417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.658207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.659797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1063/2122] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:57 2022] DELETE /identity/v3/projects/c5371b99224a4a3e92f46cb440945692 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:57.663789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:57.673512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e013-578f-4f2a-8a1c-d2a9110c4c31 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:57.699077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:57.731053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:57.731655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:57.731932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:57.732202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] PATH_INFO: `/v3/users/367de0a436c746178981fefa1b5d6649` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:57.733315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367de0a436c746178981fefa1b5d6649)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:57.735555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:57.781115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367de0a436c746178981fefa1b5d6649'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.781999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367de0a436c746178981fefa1b5d6649'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.782616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367de0a436c746178981fefa1b5d6649'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.789725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367de0a436c746178981fefa1b5d6649'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.801619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367de0a436c746178981fefa1b5d6649'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:57.814254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1060/2123] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:57 2022] DELETE /identity/v3/users/367de0a436c746178981fefa1b5d6649 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:58.097402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:58.098998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:58.118359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538692-2e3e-41fd-8829-7cf89e0f1333 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:58.179073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:58.179736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.180013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.180316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] PATH_INFO: `/v3/projects/ad76ddacc9f24f3887f3a46acc1f7323` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.181403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad76ddacc9f24f3887f3a46acc1f7323)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:58.183366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:58.236147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad76ddacc9f24f3887f3a46acc1f7323', 'request_id': 'req-ef36b880-c57c-483f-841c-01bbdff3fd9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.241696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad76ddacc9f24f3887f3a46acc1f7323', 'request_id': 'req-ef36b880-c57c-483f-841c-01bbdff3fd9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.247904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad76ddacc9f24f3887f3a46acc1f7323', 'request_id': 'req-ef36b880-c57c-483f-841c-01bbdff3fd9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.261483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] The token cache is being invalidate because project ad76ddacc9f24f3887f3a46acc1f7323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:58.261765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.262373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.263895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1064/2124] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:58 2022] DELETE /identity/v3/projects/ad76ddacc9f24f3887f3a46acc1f7323 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:58.267899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:58.269281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:58.281698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4c46-62b0-42c7-9a8a-fe4205b28a81 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:58.337329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:58.337954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.338235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.338510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] PATH_INFO: `/v3/users/de735818b1c94f5eb3e4efde2f0d49d1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.339704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de735818b1c94f5eb3e4efde2f0d49d1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:58.341557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:58.386600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de735818b1c94f5eb3e4efde2f0d49d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.387276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de735818b1c94f5eb3e4efde2f0d49d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.388006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de735818b1c94f5eb3e4efde2f0d49d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.390515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:58.396205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de735818b1c94f5eb3e4efde2f0d49d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.409040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-bf7db4de-1e3b-4c94-964f-ff34a9f3fd0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de735818b1c94f5eb3e4efde2f0d49d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.421177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1061/2125] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:58 2022] DELETE /identity/v3/users/de735818b1c94f5eb3e4efde2f0d49d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:58.688246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:58.689680 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:58.702267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36b880-c57c-483f-841c-01bbdff3fd9c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:58.710463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:58.762621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:58.763281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.763588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.763864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] PATH_INFO: `/v3/projects/b4e3a7548deb43948c3aa2b573d626db` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.764980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e3a7548deb43948c3aa2b573d626db)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:58.767155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:58.813214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e3a7548deb43948c3aa2b573d626db', 'request_id': 'req-28196267-e92e-4dca-b568-0675b69e4bd6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.818608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e3a7548deb43948c3aa2b573d626db', 'request_id': 'req-28196267-e92e-4dca-b568-0675b69e4bd6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.824681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e3a7548deb43948c3aa2b573d626db', 'request_id': 'req-28196267-e92e-4dca-b568-0675b69e4bd6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.837670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] The token cache is being invalidate because project b4e3a7548deb43948c3aa2b573d626db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:58.837959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.838584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-28196267-e92e-4dca-b568-0675b69e4bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:58.840101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1065/2126] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:58 2022] DELETE /identity/v3/projects/b4e3a7548deb43948c3aa2b573d626db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:58.846883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.847345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.847436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.867075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:58.868231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:58.931409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1062/2127] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:58.941097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.941384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.941662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.961846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:58.963083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:58.972772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1066/2128] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:58.977066 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:58.986978 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6006bf-e8cf-4339-8793-a0bbf6c3aac7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:58.991546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:58.992143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:58.992425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:58.992701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:58.993375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:38:58.993680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:58.995336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.007056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1063/2129] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:38:59.011383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.022499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bf557-edbc-48eb-b5f9-194655b98d4a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.027280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.027876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.028145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.028413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.029051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.031002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.067914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.068981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1067/2130] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:59.082945 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5262a-1e96-4e3f-9f99-5ba53107546e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.156905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.157659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.157943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.158219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.159019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.160931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.188387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1064/2131] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:59.190145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.201097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e61ec-829d-4448-8cd2-3eebd06166c1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.205760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.206348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.206628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.206929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.207569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.209170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.219511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1068/2132] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:59.223428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.233535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475275-0ed6-4225-9e24-d108e146abd3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.238890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.239499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.239811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.240114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.240766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.242633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae41e5-d8cb-426b-86a8-970a6338d9a9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.253376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1065/2133] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:38:59.257541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.267684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca597-8909-4014-8b0e-833f55a61465 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.272483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.273235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.273515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.273802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] PATH_INFO: `/v3/projects/3277b5d3770f40d88fed2d5245f13058/users/0f79c9bb000043aa80ee04e86e928151/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.289097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3277b5d3770f40d88fed2d5245f13058, user_id=0f79c9bb000043aa80ee04e86e928151, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.291911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.306326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1069/2134] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:59 2022] PUT /identity/v3/projects/3277b5d3770f40d88fed2d5245f13058/users/0f79c9bb000043aa80ee04e86e928151/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:59.317836 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.318121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.318396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.347269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] MFA Rules not processed for user `0f79c9bb000043aa80ee04e86e928151`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:59.348558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:59.369020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:59.414514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1066/2135] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:59.419126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.461178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:59.484122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350ebe-fc06-4997-93fc-6fa602f810bb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.489451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.490047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.490332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.490608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.491267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.493169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.522090 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1070/2136] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:59.526143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.537337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e24d1-4e6e-438b-a995-0648cea84bf5 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.592754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.593355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.593702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.594439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.595190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.597365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.617548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1067/2137] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:38:59.621637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.632024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e65-1dee-453e-9efb-d6d54dc70be8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.636892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.637487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.637768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.638041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.638720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.640349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.652990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1071/2138] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:59.657564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.667692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955485-5533-45a3-bb26-f6362b587d78 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.672191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.672831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.673106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.673378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] PATH_INFO: `/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30/users/c44fa52d45ec47e9ad3a8af1c7e8194e/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.687785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c669c9a65fb34d1883bd7e0d6ac4ba30, user_id=c44fa52d45ec47e9ad3a8af1c7e8194e, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.690517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.704491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1068/2139] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:59 2022] PUT /identity/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30/users/c44fa52d45ec47e9ad3a8af1c7e8194e/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:59.708451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.718355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3b9bf-7e3e-4c95-8d1c-31a4bb45f95b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.774360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.775019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.775301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.775575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.776293 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.777950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798fa24-0d23-41fb-a5b8-2c37022444a9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.788841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1072/2140] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:38:59.792834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.803234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547610-dfda-4424-8ab4-5067f1c3757d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:59.807933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:59.808583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.808868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.809143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] PATH_INFO: `/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30/users/c44fa52d45ec47e9ad3a8af1c7e8194e/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.811807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c669c9a65fb34d1883bd7e0d6ac4ba30, user_id=c44fa52d45ec47e9ad3a8af1c7e8194e, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:59.814515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:59.824918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1069/2141] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:38:59 2022] PUT /identity/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30/users/c44fa52d45ec47e9ad3a8af1c7e8194e/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:38:59.835693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:59.836324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:59.836610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:59.867843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] MFA Rules not processed for user `c44fa52d45ec47e9ad3a8af1c7e8194e`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:38:59.869083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:59.952435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1073/2142] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:38:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:38:59.963975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:59.965725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:00.031839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ad9-4fe5-4187-b7e9-04d79cde77d9 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:00.035467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:00.036097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:00.036376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:00.036644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:00.045361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:00.046948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86231115-4be7-444d-bd0e-3b6a848d34ce service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:00.056014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1070/2143] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:00.536045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:00.570416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:00.598621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d717-3c1c-4580-9ec4-794f8eabb55f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:00.603467 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:00.604386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:00.604775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:00.605055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] PATH_INFO: `/v3/users/0f79c9bb000043aa80ee04e86e928151` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:00.606245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f79c9bb000043aa80ee04e86e928151)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:00.608669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:00.660589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f79c9bb000043aa80ee04e86e928151'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:00.661244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f79c9bb000043aa80ee04e86e928151'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:00.668282 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f79c9bb000043aa80ee04e86e928151'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:00.668986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f79c9bb000043aa80ee04e86e928151'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:00.676903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f79c9bb000043aa80ee04e86e928151'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:00.694898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1074/2144] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:00 2022] DELETE /identity/v3/users/0f79c9bb000043aa80ee04e86e928151 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:00.699458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:00.699750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:00.700032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:00.719835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:00.721040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:00.733071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:00.787284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1071/2145] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:00.797790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:00.799248 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:00.856717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5236246-5231-48d7-b4e3-8b9fe7b3dbe5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:00.861427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:00.862042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:00.862323 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:00.862598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:00.873387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:00.874905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:00.885466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1075/2146] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:01.173181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:01.174738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:01.188921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f445-3b78-4802-8ca5-aa2505b2ad57 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:01.196170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:01.196170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:01.196170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:01.196170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] PATH_INFO: `/v3/projects/3277b5d3770f40d88fed2d5245f13058` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:01.196852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3277b5d3770f40d88fed2d5245f13058)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:01.199254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:01.257822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3277b5d3770f40d88fed2d5245f13058', 'request_id': 'req-fa168cc4-5be1-4752-a1fb-22163e9a27b6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.263632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3277b5d3770f40d88fed2d5245f13058', 'request_id': 'req-fa168cc4-5be1-4752-a1fb-22163e9a27b6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.270503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3277b5d3770f40d88fed2d5245f13058', 'request_id': 'req-fa168cc4-5be1-4752-a1fb-22163e9a27b6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.284319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] The token cache is being invalidate because project 3277b5d3770f40d88fed2d5245f13058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:01.284609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.285241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.286776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1072/2147] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:01 2022] DELETE /identity/v3/projects/3277b5d3770f40d88fed2d5245f13058 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:01.292351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:01.303729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138d66b-6da9-4a73-8f99-0597a43e2601 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:01.380133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:01.395243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:01.395911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:01.396193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:01.396519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] PATH_INFO: `/v3/users/c44fa52d45ec47e9ad3a8af1c7e8194e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:01.397687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44fa52d45ec47e9ad3a8af1c7e8194e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:01.399693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:01.451962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44fa52d45ec47e9ad3a8af1c7e8194e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.452962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44fa52d45ec47e9ad3a8af1c7e8194e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.468008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44fa52d45ec47e9ad3a8af1c7e8194e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.470239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44fa52d45ec47e9ad3a8af1c7e8194e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.480725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c62ec09e-943d-46d2-9132-46be6088f0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44fa52d45ec47e9ad3a8af1c7e8194e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.507761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1076/2148] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:01 2022] DELETE /identity/v3/users/c44fa52d45ec47e9ad3a8af1c7e8194e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:01.821010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:01.822663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:01.836907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa168cc4-5be1-4752-a1fb-22163e9a27b6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:01.901929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:01.902561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:01.912974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:01.912974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] PATH_INFO: `/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:01.912974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c669c9a65fb34d1883bd7e0d6ac4ba30)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:01.912974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:01.953550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:01.957812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c669c9a65fb34d1883bd7e0d6ac4ba30', 'request_id': 'req-a04f6a88-4a64-4179-a242-9c7f55e7289e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.965744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c669c9a65fb34d1883bd7e0d6ac4ba30', 'request_id': 'req-a04f6a88-4a64-4179-a242-9c7f55e7289e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.975019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c669c9a65fb34d1883bd7e0d6ac4ba30', 'request_id': 'req-a04f6a88-4a64-4179-a242-9c7f55e7289e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.988402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] The token cache is being invalidate because project c669c9a65fb34d1883bd7e0d6ac4ba30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:01.988687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.989320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a04f6a88-4a64-4179-a242-9c7f55e7289e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:01.990927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1073/2149] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:01 2022] DELETE /identity/v3/projects/c669c9a65fb34d1883bd7e0d6ac4ba30 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:01.995920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:01.996407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:01.996789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.018227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:02.019491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.079173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1077/2150] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:02.086968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.087247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.087623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.107393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:02.108592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.117080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1074/2151] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:02.121710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.135133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7d29-3d7a-4ee9-ba0f-93fdcc6323b6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.141356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.142032 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.142340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.142617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.143470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:02.143777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.145433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.157690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1078/2152] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:02.162133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.171509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472831ff-1c70-4b91-ade7-2bbecfeef8af None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.176813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.177392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.177663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.177961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.178581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.180500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.208120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1075/2153] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:02.212241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.222954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff693339-e87a-4d02-8e6c-b54baef4047a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.279319 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.279801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.280085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.280363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.280999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.283541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.296340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:02.313311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1079/2154] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:02.317543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.329190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db07347-df05-4850-b570-3a077c85401b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.333802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.334418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.334731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.335003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.335641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.337243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.348201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1076/2155] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:02.352190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.363059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65436286-5ee2-4a61-ae08-b41abd87db51 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.367817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.368412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.368693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.368994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.369626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.371272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27255d-5f3b-4676-b24c-1da172fda330 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.383307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1080/2156] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:02.387375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.397694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b9d1f-8598-416e-9158-c1daa783fca6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.402282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.402943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.403221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.403495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] PATH_INFO: `/v3/projects/97bcb831f0ad427ca798f262e21e2397/users/b2053911fdc84a9d98922a9a8d847c42/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.418478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bcb831f0ad427ca798f262e21e2397, user_id=b2053911fdc84a9d98922a9a8d847c42, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.421312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.436796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1077/2157] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:02 2022] PUT /identity/v3/projects/97bcb831f0ad427ca798f262e21e2397/users/b2053911fdc84a9d98922a9a8d847c42/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:02.447894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.448178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.448454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.477005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] MFA Rules not processed for user `b2053911fdc84a9d98922a9a8d847c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:02.478268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.542263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1081/2158] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:02.554226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.556061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.630041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d83a4-e48e-46d5-9353-0a2fb9895d7c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.638565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.638565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.638565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.638565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.645886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.647405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.658195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1078/2159] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:02.686365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.691113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.692448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:02.797630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2cef-4ff9-4534-ac9e-12297a815c93 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.806936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.807580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.807861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.808134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] PATH_INFO: `/v3/users/b2053911fdc84a9d98922a9a8d847c42` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.809285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2053911fdc84a9d98922a9a8d847c42)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.811247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.826947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bb6ba-f0d1-4709-9d0f-c741a7730543 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:02.832751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:02.836894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:02.851209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.851209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.851209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] PATH_INFO: `/v3/users/82f70b90111f49ca859188da08044796` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.853747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f70b90111f49ca859188da08044796)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:02.854807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:02.893513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2053911fdc84a9d98922a9a8d847c42'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.894503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2053911fdc84a9d98922a9a8d847c42'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.895511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2053911fdc84a9d98922a9a8d847c42'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.905309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2053911fdc84a9d98922a9a8d847c42'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.919465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6f79ace0-e80b-409c-8e96-13202541d660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2053911fdc84a9d98922a9a8d847c42'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.923138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f70b90111f49ca859188da08044796'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.924356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f70b90111f49ca859188da08044796'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.934027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1079/2160] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:02 2022] DELETE /identity/v3/users/b2053911fdc84a9d98922a9a8d847c42 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:02.934998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f70b90111f49ca859188da08044796'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.935683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f70b90111f49ca859188da08044796'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.938883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.939263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.939552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:02.946994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2514cfd7-f2f5-4afe-acea-9da276fa217b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f70b90111f49ca859188da08044796'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:02.959119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:02.963529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:02.965298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:02.979561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1082/2161] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:02 2022] DELETE /identity/v3/users/82f70b90111f49ca859188da08044796 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:02.981934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:02.982268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:02.982555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.014521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:03.015946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.056671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1080/2162] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:03.073065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.074507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.090596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1083/2163] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:03.101251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.104135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.155065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be77a-40bc-4766-8160-7ac47cbc6df9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.167543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598f850-d7b3-4418-b44e-a3d0619976fa None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.171794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.172854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.173174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.173576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.185315 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.186814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:03.198183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1084/2164] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:03.215550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:03.222462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.223549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.224093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.224784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.236083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.237529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:03.247270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1081/2165] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:03.565131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.566750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.580871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720e77b-7c6b-4a2a-872c-38242eae7d98 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.586541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.587338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.587720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.588104 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] PATH_INFO: `/v3/projects/947191f45d2749f6af46dc226da6ded6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.589368 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947191f45d2749f6af46dc226da6ded6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.591892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:03.653152 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.653856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947191f45d2749f6af46dc226da6ded6', 'request_id': 'req-8b8513fd-6787-4af3-93cf-7673b67dc5b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.654623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.660394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947191f45d2749f6af46dc226da6ded6', 'request_id': 'req-8b8513fd-6787-4af3-93cf-7673b67dc5b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.668199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947191f45d2749f6af46dc226da6ded6', 'request_id': 'req-8b8513fd-6787-4af3-93cf-7673b67dc5b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.668895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf54e-4dd6-42a6-9916-d83730252986 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.684234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] The token cache is being invalidate because project 947191f45d2749f6af46dc226da6ded6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:03.684599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.685298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b8513fd-6787-4af3-93cf-7673b67dc5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.687145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1085/2166] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:03 2022] DELETE /identity/v3/projects/947191f45d2749f6af46dc226da6ded6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:03.692169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a9ad7-3f94-486c-901e-bd17d91492f9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.692459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a9ad7-3f94-486c-901e-bd17d91492f9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.692746 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a9ad7-3f94-486c-901e-bd17d91492f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.712520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-253a9ad7-3f94-486c-901e-bd17d91492f9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:03.713736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-253a9ad7-3f94-486c-901e-bd17d91492f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.724144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:03.737557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.738189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.738468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.738776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] PATH_INFO: `/v3/projects/97bcb831f0ad427ca798f262e21e2397` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.739925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bcb831f0ad427ca798f262e21e2397)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.741846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:03.784090 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1086/2167] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:03.793220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.793641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.793923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.801223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bcb831f0ad427ca798f262e21e2397', 'request_id': 'req-e8c53e96-e562-458c-a4ee-0982e10a1a16'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.806884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bcb831f0ad427ca798f262e21e2397', 'request_id': 'req-e8c53e96-e562-458c-a4ee-0982e10a1a16'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.813724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bcb831f0ad427ca798f262e21e2397', 'request_id': 'req-e8c53e96-e562-458c-a4ee-0982e10a1a16'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.814024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:03.815272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:03.826802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] The token cache is being invalidate because project 97bcb831f0ad427ca798f262e21e2397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:03.827088 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.827714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:03.829474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1082/2168] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:03 2022] DELETE /identity/v3/projects/97bcb831f0ad427ca798f262e21e2397 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:03.898802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1087/2169] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:03.900527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.918154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c53e96-e562-458c-a4ee-0982e10a1a16 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.926383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.927114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.927448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.927730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.928410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:03.928926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.930776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:03.949849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1083/2170] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:03.951102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:03.961672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bc643-a498-463f-bb1a-937df41c3147 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:03.966864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:03.967499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:03.967779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:03.968209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:03.970369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:03.971903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.009960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1088/2171] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:03 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:04.014001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.025990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247490d8-75d1-4cf8-bbb2-3a2cab9cf39f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.100069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.101254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.101550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.101831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.102491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.104673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.117233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:04.131308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1084/2172] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:04 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:04.133768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.145260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23d31a-2964-4cb1-88fc-2a25014983fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.150000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.150626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.150946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.151224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.151874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.153474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.166145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1089/2173] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:04.170142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.181385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbb3d-7c7c-4e75-aeb2-1ab74658e166 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.186279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.186916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.187362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.187706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.188361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.189961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452de9-51dd-43dc-a32a-18bec1f0a2b0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.200942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1085/2174] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:04.205957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.216219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b9bd3-4e65-4418-9693-0104e9156832 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.221018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.221921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.222200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.222475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] PATH_INFO: `/v3/projects/5dda94b5023d4f62b492e175a4217944/users/6f402bc1ffd94858ad8d4958a7e7e841/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.245989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dda94b5023d4f62b492e175a4217944, user_id=6f402bc1ffd94858ad8d4958a7e7e841, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.248843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.264610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1090/2175] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:04 2022] PUT /identity/v3/projects/5dda94b5023d4f62b492e175a4217944/users/6f402bc1ffd94858ad8d4958a7e7e841/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:04.275719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.276232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.276325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.306053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] MFA Rules not processed for user `6f402bc1ffd94858ad8d4958a7e7e841`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:04.307417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:04.387997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1086/2176] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:04.389402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.491778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec57fb-cf5d-474c-9246-a22719123d1e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.496925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.497787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.498274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.498625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.499638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.502553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.508108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:04.533322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1091/2177] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:04 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:04.537712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.550482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ed87-32f4-4841-8cd6-12217e877bb8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.608506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.609022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.609963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.610477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.611134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.613041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.634882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1087/2178] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:04 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:04.639238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.649405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7392f5-7bc7-4b61-b25d-cc0405393cd7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.654037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.654626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.654936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.655208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.655851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.658031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.668030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1092/2179] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:04.672784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.682475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc604cc-85b6-433f-b8c6-2a0ae583f52d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.687622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.688457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.688738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.689015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] PATH_INFO: `/v3/projects/b423a264bc724bb98e535a4e0ba77424/users/12d5a823d768482199849f77199074c9/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.704437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b423a264bc724bb98e535a4e0ba77424, user_id=12d5a823d768482199849f77199074c9, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.707118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.720704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1088/2180] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:04 2022] PUT /identity/v3/projects/b423a264bc724bb98e535a4e0ba77424/users/12d5a823d768482199849f77199074c9/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:04.725045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.735717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ef550-5218-4386-af87-067952c1587c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.801451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.802084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.802364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.802657 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.803313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.804931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc02ebe-5d4a-4a87-9d15-04e3d736a3dc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.814875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1093/2181] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:04.819175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.829890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0820b4-6cc7-490f-a2de-80e6de7b4bfe admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:04.834467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:04.835154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.835435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.835709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] PATH_INFO: `/v3/projects/b423a264bc724bb98e535a4e0ba77424/users/12d5a823d768482199849f77199074c9/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.837623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b423a264bc724bb98e535a4e0ba77424, user_id=12d5a823d768482199849f77199074c9, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:04.840294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:04.851113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1089/2182] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:04 2022] PUT /identity/v3/projects/b423a264bc724bb98e535a4e0ba77424/users/12d5a823d768482199849f77199074c9/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:04.861307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:04.861530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:04.861806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:04.889566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] MFA Rules not processed for user `12d5a823d768482199849f77199074c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:04.891551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:04.967828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1094/2183] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:04.980473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:04.982195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:05.013349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:05.041729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60630-6c56-4365-9b30-e36ecb33e5ae admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:05.045467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:05.046055 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:05.046367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:05.046760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:05.055650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:05.057067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:05.066353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1090/2184] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:05.156841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:05.166903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98dd1-137d-463b-a5a1-0246ef75dc9e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:05.170607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:05.171324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:05.171758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:05.172298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:05.230294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:05.235220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:05.236784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa96d1-2bfd-47fb-bcda-1aa549a3ba01 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:05.246709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1095/2185] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:15.062099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=14 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:15.242414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:16.962473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:17.032618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e5cce-8c78-4117-ad22-f8d42c4c0669 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:17.037298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:17.037937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:17.038219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:17.038495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] PATH_INFO: `/v3/users/6f402bc1ffd94858ad8d4958a7e7e841` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:17.039669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f402bc1ffd94858ad8d4958a7e7e841)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:17.041709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:17.101942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f402bc1ffd94858ad8d4958a7e7e841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.102663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f402bc1ffd94858ad8d4958a7e7e841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.103320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f402bc1ffd94858ad8d4958a7e7e841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.113077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f402bc1ffd94858ad8d4958a7e7e841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.126592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f402bc1ffd94858ad8d4958a7e7e841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.146605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1091/2186] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:16 2022] DELETE /identity/v3/users/6f402bc1ffd94858ad8d4958a7e7e841 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:17.152304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:17.152776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:17.152868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:17.172664 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:17.176243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:17.271106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1096/2187] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:17.280904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:17.283170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:17.367957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c775c-e8bd-4f99-bfd2-d4d55e6def0c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:17.372127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:17.372621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:17.372903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:17.373177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:17.383104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:17.384561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:17.394534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1092/2188] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:17.667743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:17.669169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:17.681865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e35de0-64e5-43bc-b9be-d63292fc4766 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:17.686826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:17.687466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:17.687792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:17.688085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] PATH_INFO: `/v3/projects/5dda94b5023d4f62b492e175a4217944` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:17.690091 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dda94b5023d4f62b492e175a4217944)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:17.693403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:17.739954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dda94b5023d4f62b492e175a4217944', 'request_id': 'req-7be4fdba-8303-463c-b0bd-540c1e21a941'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.745627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dda94b5023d4f62b492e175a4217944', 'request_id': 'req-7be4fdba-8303-463c-b0bd-540c1e21a941'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.752331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dda94b5023d4f62b492e175a4217944', 'request_id': 'req-7be4fdba-8303-463c-b0bd-540c1e21a941'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.764227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] The token cache is being invalidate because project 5dda94b5023d4f62b492e175a4217944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:17.764543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.765162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.766684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1097/2189] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:17 2022] DELETE /identity/v3/projects/5dda94b5023d4f62b492e175a4217944 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:17.770622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:17.780005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc75c3-39af-41af-88fd-6ada7ed9b3cb service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:17.798425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:17.838504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:17.839075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:17.839351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:17.839622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] PATH_INFO: `/v3/users/12d5a823d768482199849f77199074c9` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:17.840735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d5a823d768482199849f77199074c9)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:17.842668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:17.893357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d5a823d768482199849f77199074c9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.894024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d5a823d768482199849f77199074c9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.894661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d5a823d768482199849f77199074c9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.902515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d5a823d768482199849f77199074c9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.916063 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c07f63e5-85b1-41d7-a3f3-f66001a95563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d5a823d768482199849f77199074c9'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:17.928924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1093/2190] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:17 2022] DELETE /identity/v3/users/12d5a823d768482199849f77199074c9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:18.218337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.218337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:18.234195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4fdba-8303-463c-b0bd-540c1e21a941 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.296418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.297005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.297276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.297564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] PATH_INFO: `/v3/projects/b423a264bc724bb98e535a4e0ba77424` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.298799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b423a264bc724bb98e535a4e0ba77424)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.300720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.332452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:18.350540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b423a264bc724bb98e535a4e0ba77424', 'request_id': 'req-d386b621-d0e2-415d-a0f3-c1282f077958'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:18.357022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b423a264bc724bb98e535a4e0ba77424', 'request_id': 'req-d386b621-d0e2-415d-a0f3-c1282f077958'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:18.363722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b423a264bc724bb98e535a4e0ba77424', 'request_id': 'req-d386b621-d0e2-415d-a0f3-c1282f077958'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:18.377486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] The token cache is being invalidate because project b423a264bc724bb98e535a4e0ba77424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:18.377486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:18.378101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d386b621-d0e2-415d-a0f3-c1282f077958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:18.379758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1098/2191] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:18 2022] DELETE /identity/v3/projects/b423a264bc724bb98e535a4e0ba77424 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:18.384568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.385022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.385113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.411161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:18.412645 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:18.487889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1094/2192] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:18.496757 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.497072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.497354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.539537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:18.540890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:18.549497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1099/2193] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:18.553741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.564331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f15074-a6fe-41eb-b421-cd6e28c04f5e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.569153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.569755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.570035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.570307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.570984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:18.571321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.573097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.595575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1095/2194] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:18.597140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.606446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1a65-bed9-44c1-a287-badd8a0d88d7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.616298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.616912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.617234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.617998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.618635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.621375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.651504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1100/2195] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:18 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:18.655596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.666415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070839d-3893-4c8e-bbd5-bdffde03abf3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.693530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:18.733820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.734437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.734752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.735029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.736293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.738226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.759515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1096/2196] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:18 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:18.763613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.774074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fe0da-3345-4d30-9961-cb917ff032cc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.778690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.779294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.779561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.779836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.780478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.782612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.792649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1101/2197] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:18.796665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.807122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccabd15-f9fa-4c23-ad74-c9164e8f0969 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.811700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.812371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.812650 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.812925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.813615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.815228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e51a1-8102-4084-b02e-68ff25af458b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.825889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1097/2198] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:18.830031 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:18.840533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3828-82a1-4907-af62-e5d41ab939af admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:18.846286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:18.847313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.847600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.847880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] PATH_INFO: `/v3/projects/8ed93b3016f04352a998e64a8f5bdace/users/77a897fdcf4843368a126041eceb8e59/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.868609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed93b3016f04352a998e64a8f5bdace, user_id=77a897fdcf4843368a126041eceb8e59, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:18.871536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:18.885819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1102/2199] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:18 2022] PUT /identity/v3/projects/8ed93b3016f04352a998e64a8f5bdace/users/77a897fdcf4843368a126041eceb8e59/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:18.896255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:18.896544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:18.896818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:18.929450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] MFA Rules not processed for user `77a897fdcf4843368a126041eceb8e59`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:18.930634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:19.003611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1098/2200] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:19.008208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.073875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec8c4b-34b8-4c04-95d2-d1e982f1f7b1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.078597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.079286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.079566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.079840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.080469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.082373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.112008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1103/2201] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:19 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:19.115998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.127659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a26fa8-8641-453c-8bea-c43be08b438d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.187353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.187880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.188197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.188583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.189246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.191189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.212543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1099/2202] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:19 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:19.216555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.230782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0fb8c-98d5-4ccb-b4fd-7d92151d8e8e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.235476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.236084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.236363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.236640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.237287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.238910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.253169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1104/2203] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:19.254366 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.264768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc6b6-d585-4209-abf2-37be9aa8aa4e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.271143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.271963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.272357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.272638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] PATH_INFO: `/v3/projects/78e242efd7fe4fd081536d01959c5a6f/users/43b859a6a24e4a979bad0f01a9e7ea86/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.289384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e242efd7fe4fd081536d01959c5a6f, user_id=43b859a6a24e4a979bad0f01a9e7ea86, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.292309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.306772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1100/2204] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:19 2022] PUT /identity/v3/projects/78e242efd7fe4fd081536d01959c5a6f/users/43b859a6a24e4a979bad0f01a9e7ea86/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:19.310891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.325697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adb10e-58e5-4196-98d0-40c20e584a33 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.351847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:19.388885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.389482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.389763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.390113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.390718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.393086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7723e74-b0e2-434c-a1b7-ac99ae3186f5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.404255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1105/2205] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:19.412085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.417751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:19.424472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9049dc-7e58-4751-beab-cc064213a3d0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.430176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.430863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.431149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.431431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] PATH_INFO: `/v3/projects/78e242efd7fe4fd081536d01959c5a6f/users/43b859a6a24e4a979bad0f01a9e7ea86/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.433374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e242efd7fe4fd081536d01959c5a6f, user_id=43b859a6a24e4a979bad0f01a9e7ea86, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.436067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.448333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1101/2206] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:19 2022] PUT /identity/v3/projects/78e242efd7fe4fd081536d01959c5a6f/users/43b859a6a24e4a979bad0f01a9e7ea86/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:39:19.461178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.461479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.461758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.495084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] MFA Rules not processed for user `43b859a6a24e4a979bad0f01a9e7ea86`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:19.496718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:19.575679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1106/2207] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:19.587076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:19.588858 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:19.649078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb34fa-ba11-4c22-bb8b-47fc4f835acc admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:19.652745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:19.653347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:19.653626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:19.653897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:19.669402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:19.671018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830c133-ece4-42b7-b01f-856fa255a483 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:19.682286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1102/2208] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:21.432618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:21.494803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d563f5-dff4-4b2a-9dfc-aa299b49328e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:21.499609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:21.500240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:21.500527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:21.500806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] PATH_INFO: `/v3/users/77a897fdcf4843368a126041eceb8e59` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:21.502598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a897fdcf4843368a126041eceb8e59)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:21.504644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:21.550232 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a897fdcf4843368a126041eceb8e59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:21.550846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a897fdcf4843368a126041eceb8e59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:21.552040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:21.560015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a897fdcf4843368a126041eceb8e59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:21.560721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a897fdcf4843368a126041eceb8e59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:21.571402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a897fdcf4843368a126041eceb8e59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:21.592213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1107/2209] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:21 2022] DELETE /identity/v3/users/77a897fdcf4843368a126041eceb8e59 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:21.597404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:21.597925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:21.598053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:21.616747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:21.617936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:21.675001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1103/2210] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:21.687186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:21.689203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:21.755024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f64b9-ad78-42ac-b80c-add31a01008f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:21.758706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:21.759307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:21.759590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:21.759881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:21.768945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:21.770372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:21.780028 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1108/2211] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:22.047677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.049344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.061265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189768ca-ad3b-4740-afcf-a9d335aa40c3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.066028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.066669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.066990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.067258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] PATH_INFO: `/v3/users/bd10d229bed14abf9bf2ec85b7e4bac0` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.068755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd10d229bed14abf9bf2ec85b7e4bac0)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.070668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.116188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.118431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.133931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd10d229bed14abf9bf2ec85b7e4bac0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.134763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd10d229bed14abf9bf2ec85b7e4bac0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.135402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd10d229bed14abf9bf2ec85b7e4bac0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.135780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6791aa4-b60e-4b77-b0ed-063bc8d3b408 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.146555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.147768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.148257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.148362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd10d229bed14abf9bf2ec85b7e4bac0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.148689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] PATH_INFO: `/v3/projects/8ed93b3016f04352a998e64a8f5bdace` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.150985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed93b3016f04352a998e64a8f5bdace)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.154741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.170080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0436a0ee-6aa6-4ae2-9828-26fef3d9f10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd10d229bed14abf9bf2ec85b7e4bac0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.193768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1104/2212] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:22 2022] DELETE /identity/v3/users/bd10d229bed14abf9bf2ec85b7e4bac0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:22.196886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.197127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.197410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.205969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:22.218456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:22.220562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.231733 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed93b3016f04352a998e64a8f5bdace', 'request_id': 'req-b3f54233-fe88-4481-b994-4bec423120b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.237328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed93b3016f04352a998e64a8f5bdace', 'request_id': 'req-b3f54233-fe88-4481-b994-4bec423120b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.243993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed93b3016f04352a998e64a8f5bdace', 'request_id': 'req-b3f54233-fe88-4481-b994-4bec423120b7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.257314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] The token cache is being invalidate because project 8ed93b3016f04352a998e64a8f5bdace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:22.257602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.258264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.259916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1109/2213] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:22 2022] DELETE /identity/v3/projects/8ed93b3016f04352a998e64a8f5bdace => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:22.264111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.265477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.286857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1105/2214] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:22.296987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.298482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.327489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:22.343540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54233-fe88-4481-b994-4bec423120b7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.349647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.350357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.350835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.351181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] PATH_INFO: `/v3/users/43b859a6a24e4a979bad0f01a9e7ea86` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.352371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b859a6a24e4a979bad0f01a9e7ea86)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.354420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.376775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c7fc-869b-4d9a-b8f8-9f011fdafd06 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.380688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.381416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.381691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.382097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.392313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.393763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.404357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1106/2215] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:22.413087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b859a6a24e4a979bad0f01a9e7ea86'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.414224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b859a6a24e4a979bad0f01a9e7ea86'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.423590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b859a6a24e4a979bad0f01a9e7ea86'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.424362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b859a6a24e4a979bad0f01a9e7ea86'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.434803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b859a6a24e4a979bad0f01a9e7ea86'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.457923 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1110/2216] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:22 2022] DELETE /identity/v3/users/43b859a6a24e4a979bad0f01a9e7ea86 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:39:22.786083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.796560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1cbfd-aca0-42e0-8cba-3a3b5fc68c93 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.835583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:22.837096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:22.881222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.881854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.882135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.882409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] PATH_INFO: `/v3/projects/78e242efd7fe4fd081536d01959c5a6f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.883720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e242efd7fe4fd081536d01959c5a6f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.885992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.923941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d478d-f5ee-4d31-9d3b-cf85c09c966c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:22.928856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:22.929496 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.929775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.930047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] PATH_INFO: `/v3/projects/a043bf3e9d454ea0a576446ae005249f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.931623 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a043bf3e9d454ea0a576446ae005249f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:22.934484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:22.942384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e242efd7fe4fd081536d01959c5a6f', 'request_id': 'req-84a56c57-0767-4c0d-8d82-e6cb828e894d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.943698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:22.949413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e242efd7fe4fd081536d01959c5a6f', 'request_id': 'req-84a56c57-0767-4c0d-8d82-e6cb828e894d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.957317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e242efd7fe4fd081536d01959c5a6f', 'request_id': 'req-84a56c57-0767-4c0d-8d82-e6cb828e894d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.970278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] The token cache is being invalidate because project 78e242efd7fe4fd081536d01959c5a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:22.970572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.971384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-84a56c57-0767-4c0d-8d82-e6cb828e894d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.973130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1107/2217] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:22 2022] DELETE /identity/v3/projects/78e242efd7fe4fd081536d01959c5a6f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:22.977936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfae60b-ba30-4f94-abbc-7e39634b5ce2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:22.978220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfae60b-ba30-4f94-abbc-7e39634b5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:22.978494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfae60b-ba30-4f94-abbc-7e39634b5ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:22.985929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a043bf3e9d454ea0a576446ae005249f', 'request_id': 'req-208fa456-e782-4d75-b91f-6e95be92ddee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.991659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a043bf3e9d454ea0a576446ae005249f', 'request_id': 'req-208fa456-e782-4d75-b91f-6e95be92ddee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.997822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a043bf3e9d454ea0a576446ae005249f', 'request_id': 'req-208fa456-e782-4d75-b91f-6e95be92ddee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:22.997990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8bfae60b-ba30-4f94-abbc-7e39634b5ce2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:22.999223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8bfae60b-ba30-4f94-abbc-7e39634b5ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.007463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:23.012086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] The token cache is being invalidate because project a043bf3e9d454ea0a576446ae005249f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:23.012907 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:23.013691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-208fa456-e782-4d75-b91f-6e95be92ddee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:23.015429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1111/2218] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:22 2022] DELETE /identity/v3/projects/a043bf3e9d454ea0a576446ae005249f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:39:23.020811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063465f3-c5c2-4d6e-82e0-5f90b6c5a0f8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.021263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063465f3-c5c2-4d6e-82e0-5f90b6c5a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.021647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063465f3-c5c2-4d6e-82e0-5f90b6c5a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.044260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-063465f3-c5c2-4d6e-82e0-5f90b6c5a0f8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:23.045485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-063465f3-c5c2-4d6e-82e0-5f90b6c5a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.071231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1108/2219] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.079509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.080090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.080327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.105577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:23.106903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.115639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1109/2220] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.120045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.120515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1112/2221] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.129243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.129532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.129811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.129923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef33f-8be5-4a96-bc49-fac599fb78b3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.134628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.135265 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.135545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.135818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.136468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:23.136776 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.138372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.149763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:23.150627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1110/2222] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:39:23.154097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.154895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.160730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1113/2223] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.164615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.165721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6f0b9-efd2-4898-8471-04fad664bfb3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.172922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.173549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27e74-fec6-430d-9061-1198c1476341 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.173667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.174002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.174284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.178415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.178415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.178667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.179065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.179359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.179625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.180288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:23.180596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.182748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.200543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1114/2224] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:23.201612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.213482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3505ec-21f8-44a7-85e8-2f0660204ded admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.219008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.220089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.220507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.220784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.221425 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.223654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.223766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1111/2225] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:23.228154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.254721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1115/2226] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:23.258959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.299669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534c77-9535-42bf-a7b2-67ba19145f37 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.338377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafecb6-53f7-4782-9290-4593d124c63e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.345969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.346599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.347210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.347488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.348134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.350092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.365183 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:23.376543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1116/2227] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:23.380940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.387022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.387668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.387950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.388225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.388856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.390790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.400670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fbeb-a51f-4301-a115-9f54cd0ebe5b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.406222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.406988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.407816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.408089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.408731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.410372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.413209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1112/2228] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:23.417296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.423449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1117/2229] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:23.426947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.429565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bbe07-5e80-437a-b32a-6ea811afedca admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.434432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.435054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.435328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.435596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.436233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.436838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979016-816e-4c48-98bb-c415d5a6df8a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.437802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.441705 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.442366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.442739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.443387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.443827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.445407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.447807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1113/2230] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:23.451659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.456927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1118/2231] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:23.461521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.464939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73e289-9065-4ca9-aa0f-e56e17d94548 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.472866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.473752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.474134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.474680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.475658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.477806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.479642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66cc4c-bf59-4f2e-876a-62f9f6965d27 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.486975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.487651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.487997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.488216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] PATH_INFO: `/v3/projects/b41b31f46da94d3f83273a7c6bf33d89/users/f013646443584713b5bf26c7edbd3943/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.493815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1114/2232] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:23.497784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.508022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41b31f46da94d3f83273a7c6bf33d89, user_id=f013646443584713b5bf26c7edbd3943, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.508282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0167e1e-8d81-40c7-82b6-7db8c3f360ab admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.511168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173745f8-c010-4a64-a5a3-ce638e7a1c5f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.513735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.514887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.515437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.515695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] PATH_INFO: `/v3/projects/f9377ab2f80a47dc86b6f911f4cf5e40/users/ff428f2da9f443e884d12adb7ef62774/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.536655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1119/2233] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:23 2022] PUT /identity/v3/projects/b41b31f46da94d3f83273a7c6bf33d89/users/f013646443584713b5bf26c7edbd3943/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:23.538406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.538936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.539374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.542804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9377ab2f80a47dc86b6f911f4cf5e40, user_id=ff428f2da9f443e884d12adb7ef62774, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.545818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9520c2-da00-4e09-8e94-358c3d07dc66 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.550385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:23.561833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1115/2234] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:23 2022] PUT /identity/v3/projects/f9377ab2f80a47dc86b6f911f4cf5e40/users/ff428f2da9f443e884d12adb7ef62774/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:39:23.572284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.572574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.572849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.575596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] MFA Rules not processed for user `f013646443584713b5bf26c7edbd3943`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:23.577370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.605792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] MFA Rules not processed for user `ff428f2da9f443e884d12adb7ef62774`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:23.607210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.649382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1120/2235] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.664624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.666614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.684218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1116/2236] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:23.695793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:23.700869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:23.740631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf4a57-af8b-457b-9e9d-f194c5bc40ce None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.744475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.745186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.745535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.745873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.756781 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.757605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367390a1-1099-44a5-9525-137eabf29c3b service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.768379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1121/2237] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:23.786301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89619357-018b-4034-a8a6-e999eb50fa66 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:23.790835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:23.791490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:23.791719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:23.791992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:23.809213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:23.810781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:23.820185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1117/2238] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:26.915810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e3f89-d41b-406a-8c87-53dadc90e916 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:26.916412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e3f89-d41b-406a-8c87-53dadc90e916 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:26.916412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e3f89-d41b-406a-8c87-53dadc90e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:26.936000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-137e3f89-d41b-406a-8c87-53dadc90e916 None None] MFA Rules not processed for user `4dd615d7af7741b08cc44b6c983fae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:26.937210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-137e3f89-d41b-406a-8c87-53dadc90e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:27.001434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1122/2239] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:29.133746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:29.158825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:29.202544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4866-a95e-45e8-b2d9-417b1d8d2490 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:29.208775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:29.209406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:29.209690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:29.209966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] PATH_INFO: `/v3/users/ff428f2da9f443e884d12adb7ef62774` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:29.211629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff428f2da9f443e884d12adb7ef62774)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:29.213080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:29.260656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff428f2da9f443e884d12adb7ef62774'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:29.261315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff428f2da9f443e884d12adb7ef62774'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:29.261943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff428f2da9f443e884d12adb7ef62774'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:29.276169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff428f2da9f443e884d12adb7ef62774'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:29.291663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff428f2da9f443e884d12adb7ef62774'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:29.317628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1118/2240] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:29 2022] DELETE /identity/v3/users/ff428f2da9f443e884d12adb7ef62774 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:29.322138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:29.322777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:29.322777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:29.349970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:29.351747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:29.423280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:29.479915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1123/2241] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:29.491939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:29.493383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:29.570552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fb233-07cc-4b23-af3f-bde5aec55df9 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:29.578675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:29.579316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:29.579667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:29.579990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:29.591189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:29.592894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23f945-6a31-4ca1-9182-55e2f8b9ac1a service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:29.610932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1119/2242] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:30.019260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.021061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:30.041175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b92548-4bd8-4448-9430-005ccf83dd32 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.048165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.049451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.049725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.050001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] PATH_INFO: `/v3/projects/f9377ab2f80a47dc86b6f911f4cf5e40` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.053134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9377ab2f80a47dc86b6f911f4cf5e40)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.056830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.125338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9377ab2f80a47dc86b6f911f4cf5e40', 'request_id': 'req-20c6c738-11a5-43ca-8309-cb3504759ede'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:30.132827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9377ab2f80a47dc86b6f911f4cf5e40', 'request_id': 'req-20c6c738-11a5-43ca-8309-cb3504759ede'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:30.139529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9377ab2f80a47dc86b6f911f4cf5e40', 'request_id': 'req-20c6c738-11a5-43ca-8309-cb3504759ede'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:30.152928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] The token cache is being invalidate because project f9377ab2f80a47dc86b6f911f4cf5e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:30.153242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:30.154426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20c6c738-11a5-43ca-8309-cb3504759ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:30.156077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1124/2243] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:30 2022] DELETE /identity/v3/projects/f9377ab2f80a47dc86b6f911f4cf5e40 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:30.161960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.162502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.162502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.183664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:30.184880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:30.229301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:30.257354 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1120/2244] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:30.268470 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.268758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.269036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.290716 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:30.291942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:30.300154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1125/2245] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:30.304638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.315570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb898e15-a955-4257-acfb-c3f589647dd3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.323351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.324270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.324737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.325293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.326989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:30.327253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.329156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.344103 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1121/2246] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:30.349513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.367194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e40a2-7538-4299-9b77-935c22e52a9a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.381498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.382050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.382332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.382613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.383351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.385462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.415687 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1126/2247] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:30 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:30.419929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.430275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50694387-9c56-408b-9b5a-6169a7b0f0e0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.502504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.503161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.503447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.503771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.504446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.506972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.529856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1122/2248] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:30 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:30.534469 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.545337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1e9e-1cf1-4068-9fcc-f375973ab153 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.550546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.551539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.551828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.552108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.552765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.554381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.565148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1127/2249] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:30.569071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.579509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e301-1a9f-4717-a4a6-d579992de165 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.585393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.586004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.586290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.586567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.587262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.588877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8b7db-28e7-4658-8617-89591ba5be41 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.601416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1123/2250] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:30.605649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.617044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6b3dd-85b4-426a-978a-1c8df5e0c313 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.622117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.623130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.623255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.623465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] PATH_INFO: `/v3/projects/df1ee75d012f4378a607020154eaf0b4/users/ff886c5f2516476e85e9e805aa19fece/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.639938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1ee75d012f4378a607020154eaf0b4, user_id=ff886c5f2516476e85e9e805aa19fece, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.643361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.664075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1128/2251] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:30 2022] PUT /identity/v3/projects/df1ee75d012f4378a607020154eaf0b4/users/ff886c5f2516476e85e9e805aa19fece/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:30.669479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a8ea2-47ea-416f-a650-4e3ae734e853 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.669772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a8ea2-47ea-416f-a650-4e3ae734e853 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.670052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a8ea2-47ea-416f-a650-4e3ae734e853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.689259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-910a8ea2-47ea-416f-a650-4e3ae734e853 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:30.691153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-910a8ea2-47ea-416f-a650-4e3ae734e853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:30.762901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1124/2252] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:30.774212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:30.776030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:30.838878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7e24-1002-428f-aa1f-537ea4917ca7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:30.843457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:30.844153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:30.844487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:30.844964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:30.856035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:30.857880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c9de-ce51-4478-b5d1-fecaa6febc6c service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:30.868066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1129/2253] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:31.675832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:31.676276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:31.676390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:31.696783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] MFA Rules not processed for user `1541e1614d664891b1952379b003ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:31.698102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:31.739020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f53b5-9419-481b-b0bd-a673b94e14a8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:31.739568 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f53b5-9419-481b-b0bd-a673b94e14a8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:31.740478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f53b5-9419-481b-b0bd-a673b94e14a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:31.747620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:31.758299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:31.774414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1125/2254] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:31.781505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-770f53b5-9419-481b-b0bd-a673b94e14a8 None None] MFA Rules not processed for user `ff886c5f2516476e85e9e805aa19fece`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:31.782929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-770f53b5-9419-481b-b0bd-a673b94e14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:31.872702 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1130/2255] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:31.883993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:31.886584 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:31.967714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d91d1-0c80-44f4-9b56-6acd15925258 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:31.972339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:31.972960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:31.973240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:31.973514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:31.984388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:31.985729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:31.995450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1126/2256] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:33.417686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308695-0185-4830-9143-f592449b7689 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:33.418215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308695-0185-4830-9143-f592449b7689 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:33.418215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308695-0185-4830-9143-f592449b7689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:33.437788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3a308695-0185-4830-9143-f592449b7689 None None] MFA Rules not processed for user `ff886c5f2516476e85e9e805aa19fece`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:33.439029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3a308695-0185-4830-9143-f592449b7689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:33.448140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1131/2257] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:41.991072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=29 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:43.432448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=34 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:44.076852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:44.078448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:44.100250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045409f-729a-4559-8f91-96af708e9299 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:44.100389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:44.101021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:44.101317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:44.101614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] PATH_INFO: `/v3/users/3b6c0f18db3446c8a0fa88848aa322f7` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:44.102903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b6c0f18db3446c8a0fa88848aa322f7)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:44.104913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:44.170302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b6c0f18db3446c8a0fa88848aa322f7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:44.171141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b6c0f18db3446c8a0fa88848aa322f7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:44.172512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b6c0f18db3446c8a0fa88848aa322f7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:44.182874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b6c0f18db3446c8a0fa88848aa322f7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:44.204959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b6c0f18db3446c8a0fa88848aa322f7'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:44.220959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1127/2258] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:44 2022] DELETE /identity/v3/users/3b6c0f18db3446c8a0fa88848aa322f7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:44.225982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:44.226479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:44.226886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:44.249002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:44.250284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:44.318857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1132/2259] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:44.339914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:44.342462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:44.436357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af798e7-f4e2-4d2e-9868-5c96fb9cc6f3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:44.441444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:44.442301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:44.442585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:44.442891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:44.458367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:44.460871 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b4dec-f93f-438a-9d7e-707cd79b812d service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:44.474958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1128/2260] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:44.908987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:44.911173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:44.934797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e49967-f5a0-4e19-94d6-2900827eb943 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:44.940390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:44.941037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:44.941317 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:44.941590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] PATH_INFO: `/v3/projects/533cb508337149838cb6c2cf33a380a2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:44.942819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533cb508337149838cb6c2cf33a380a2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:44.944756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.014546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533cb508337149838cb6c2cf33a380a2', 'request_id': 'req-f80011c1-0dbb-427c-aeb4-abf189422502'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.021277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533cb508337149838cb6c2cf33a380a2', 'request_id': 'req-f80011c1-0dbb-427c-aeb4-abf189422502'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.031134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533cb508337149838cb6c2cf33a380a2', 'request_id': 'req-f80011c1-0dbb-427c-aeb4-abf189422502'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.071767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] The token cache is being invalidate because project 533cb508337149838cb6c2cf33a380a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:45.072083 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.073065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f80011c1-0dbb-427c-aeb4-abf189422502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.075646 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1133/2261] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:44 2022] DELETE /identity/v3/projects/533cb508337149838cb6c2cf33a380a2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:45.082064 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.082574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.083251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.128748 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:45.130160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.222923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1129/2262] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:45.231218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.231544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.231857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.259783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:45.263928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.275654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1134/2263] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:45.280482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.297169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.299339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.301404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb513e86-31bc-40bc-8f65-c0148c5b754e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.308245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.308944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.309241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.309634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.310240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:45.310568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.312370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.320651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf486c-21c6-4cfb-aa1e-0e5fb8595685 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.344249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.344249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.344249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.344249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] PATH_INFO: `/v3/users/f013646443584713b5bf26c7edbd3943` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.344659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f013646443584713b5bf26c7edbd3943)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.348157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1130/2264] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:39:45.349107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.349248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.354820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:45.360869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26ba0d-9ac4-4415-a7e1-7b4ea3c4e363 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.372987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.373806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.374190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.374587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.375382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.377655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.421524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1131/2265] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:45.432670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.447578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f013646443584713b5bf26c7edbd3943'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.448667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f013646443584713b5bf26c7edbd3943'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.460901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f013646443584713b5bf26c7edbd3943'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.464616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f013646443584713b5bf26c7edbd3943'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.478752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-43470347-a719-4ff4-aada-371342bf7d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f013646443584713b5bf26c7edbd3943'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:45.493167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:45.516514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1135/2266] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:45 2022] DELETE /identity/v3/users/f013646443584713b5bf26c7edbd3943 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:45.522585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.523063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.523459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.523637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a687-202f-45f7-b07d-04cf5fa56be7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.554151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:45.555824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.593664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.594289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.594564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.594875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.595517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.598282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.626626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1132/2267] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:45.631889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.634033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.635348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1136/2268] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:45.648503 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.648755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc900ca-9ee4-4f37-97b0-a0390a9cd70c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.649974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.654208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.654864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.655147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.655438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.656189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.657779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.669697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1133/2269] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:45.673743 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.687061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaac31-3c83-457f-b80b-7fbeff069776 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.693462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.694195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.694488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.695049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.695784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.697760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.710287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1134/2270] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:45.714689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.725742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f15dc-1c02-4b12-b3db-1e7205cc3f81 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.725959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aad4b-2e88-4d1c-b69c-1ef281bac2b1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.729755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.730359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.730772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.731027 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.731226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.731966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.732297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.732612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] PATH_INFO: `/v3/projects/0159b61c064b4eca9fcb9ed3c1fff2b5/users/0c0cd616a3794428809f28dfbc86e7b9/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.742775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.744720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31b46e-d64d-47bd-a4ab-a6339b045668 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.750364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0159b61c064b4eca9fcb9ed3c1fff2b5, user_id=0c0cd616a3794428809f28dfbc86e7b9, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.753652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.756913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1137/2271] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:45.767303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1135/2272] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:45 2022] PUT /identity/v3/projects/0159b61c064b4eca9fcb9ed3c1fff2b5/users/0c0cd616a3794428809f28dfbc86e7b9/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:39:45.773431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.773805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.774161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.794777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:45.796029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.867845 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1138/2273] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:45.878368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:45.880187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:45.925336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:45.959479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83b50e-07e8-46b9-acdd-7035c1ed9153 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:45.963107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:45.963721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:45.964003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:45.964420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:45.978306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:45.979922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7f28a-17e6-4730-95e7-9e29b2bd3622 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:45.991336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1136/2274] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:46.215927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.217465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:46.230411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff82fdf-27b7-4a4c-8fe5-0a8cb6f2ff39 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.246175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.246883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.247249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.247533 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] PATH_INFO: `/v3/projects/b41b31f46da94d3f83273a7c6bf33d89` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.248688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41b31f46da94d3f83273a7c6bf33d89)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.250686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.288047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:46.308040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41b31f46da94d3f83273a7c6bf33d89', 'request_id': 'req-ccede65b-c195-425b-9990-565c446d3af0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:46.315242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41b31f46da94d3f83273a7c6bf33d89', 'request_id': 'req-ccede65b-c195-425b-9990-565c446d3af0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:46.322811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41b31f46da94d3f83273a7c6bf33d89', 'request_id': 'req-ccede65b-c195-425b-9990-565c446d3af0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:46.340640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] The token cache is being invalidate because project b41b31f46da94d3f83273a7c6bf33d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:46.340943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:46.341707 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ccede65b-c195-425b-9990-565c446d3af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:46.343578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1139/2275] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:46 2022] DELETE /identity/v3/projects/b41b31f46da94d3f83273a7c6bf33d89 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:46.349747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.350820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.351304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.379408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:46.381023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:46.454741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1137/2276] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:46.463233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.463620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.464018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.490804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:46.493528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:46.505309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1140/2277] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:46.510028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.522750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41e93-b850-432c-a71a-c5af99200f19 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.528487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.529111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.529392 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.529717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.530444 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:46.530925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.532793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.550720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1138/2278] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:39:46.554536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.568520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84478d4-1ff6-4294-8840-b38cee87f829 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.583407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.584148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.584507 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.584756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.586229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.588427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.641799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1141/2279] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:46 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:46.643586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.656576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93fdf7-50c1-4d02-bdea-3ff57965291a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.744520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.745377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.745855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.746174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.746838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.748819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.781677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1139/2280] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:46 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:46.786010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.804926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bc00e-2341-47ad-b10d-063afdfabce2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.816332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.816959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.817249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.818350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.819162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.821719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.841927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1142/2281] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:46.846416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.862597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968744-011b-42f8-bd26-6236ba2e33a1 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.870866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.871572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.871923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.872203 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.873067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.876047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b30ff-1eb0-4570-a9ba-f98a87d8683a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.907817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.908313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1140/2282] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:46.931595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ae232-30d9-434b-bdb1-0aa89f99db26 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.940330 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.941145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.941431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.941712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] PATH_INFO: `/v3/projects/8482ef1211e34786a3a9dcb9f59dd884/users/c78766cb1cd142488488b508feba4a04/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.977511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8482ef1211e34786a3a9dcb9f59dd884, user_id=c78766cb1cd142488488b508feba4a04, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.982192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:47.000537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1143/2283] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:46 2022] PUT /identity/v3/projects/8482ef1211e34786a3a9dcb9f59dd884/users/c78766cb1cd142488488b508feba4a04/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:47.012720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:47.013015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:47.013285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:47.052526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] MFA Rules not processed for user `c78766cb1cd142488488b508feba4a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:47.053803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:47.090070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:47.138832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1141/2284] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:47.151287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:47.153465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:47.250129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188adc3-5eb2-4bcb-b18d-e52159e2d6e1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:47.256329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:47.257023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:47.257427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:47.257737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:47.268608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:47.270736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc774f-bdaf-417a-a50a-b974fbab4520 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:47.294662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1144/2285] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:39:50.195738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:50.279875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dec0a9-b650-4388-91b8-d40b17b6e40f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:50.292970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:50.293682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:50.293967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:50.294243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] PATH_INFO: `/v3/users/c78766cb1cd142488488b508feba4a04` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:50.295563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78766cb1cd142488488b508feba4a04)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:50.297525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:50.382079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78766cb1cd142488488b508feba4a04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:50.383283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78766cb1cd142488488b508feba4a04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:50.383915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78766cb1cd142488488b508feba4a04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:50.396659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78766cb1cd142488488b508feba4a04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:50.412187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78766cb1cd142488488b508feba4a04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:50.429271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1142/2286] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:39:50 2022] DELETE /identity/v3/users/c78766cb1cd142488488b508feba4a04 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:50.431879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:50.432337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:50.432945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:50.456895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:50.458370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:50.538173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:50.557454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1145/2287] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:50.571112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:50.572375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:50.637687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:50.643141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe48e-63db-4764-9105-9d1a8121cd7f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:50.649770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:50.650578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:50.651044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:50.651685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:50.665018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:50.666782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2f0b-b1b2-4fdc-93c9-a87a13e8f436 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:50.680837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1143/2288] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:51.475454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:51.477563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:51.495053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ff7-1f88-41a0-ba53-110d56ccc04f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:51.514612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:51.515382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:51.515831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:51.516183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] PATH_INFO: `/v3/projects/8482ef1211e34786a3a9dcb9f59dd884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:51.518130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8482ef1211e34786a3a9dcb9f59dd884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:51.521050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:51.666081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8482ef1211e34786a3a9dcb9f59dd884', 'request_id': 'req-550bdc7a-e9e7-400b-b970-158f3d2a0819'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:51.681521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8482ef1211e34786a3a9dcb9f59dd884', 'request_id': 'req-550bdc7a-e9e7-400b-b970-158f3d2a0819'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:51.691044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8482ef1211e34786a3a9dcb9f59dd884', 'request_id': 'req-550bdc7a-e9e7-400b-b970-158f3d2a0819'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:51.723678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] The token cache is being invalidate because project 8482ef1211e34786a3a9dcb9f59dd884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:39:51.724116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:51.732919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-550bdc7a-e9e7-400b-b970-158f3d2a0819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:39:51.743002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1146/2289] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:39:51 2022] DELETE /identity/v3/projects/8482ef1211e34786a3a9dcb9f59dd884 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:51.747856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:51.748470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:51.748652 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:51.770967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:51.772383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:51.881503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1144/2290] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:51.889107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:51.890476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:51.891486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:51.921886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:51.923331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:51.935458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1147/2291] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:51.936725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:51.949894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858dcba-6871-43f3-8052-96e91d6500c9 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:51.955230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:51.955985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:51.956273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:51.956553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:51.957224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:39:51.957534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:51.959210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:51.988051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1145/2292] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:39:51.989832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.001493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3b538-c797-4cba-b1e3-4c9354a8fd5f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.010226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.011376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.011661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.011939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.012925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.015788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.088963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1148/2293] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:51 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:52.095790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.109819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97235872-36a4-46e0-988f-b1b47146ca8a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.210012 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.211000 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.211468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.212917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.213909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.217102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.252009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1146/2294] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:52 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:52.255920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.269661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9cadf-64f1-40d4-8195-3a03b41de82d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.280550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.283969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.284737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.285192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.286888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.288775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.311616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1149/2295] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:39:52.316304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.336433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a68b6-8c27-478c-82ba-52186350ddac admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.345395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.346616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.347269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.347786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.350754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.353405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcd1ae-8aec-4af8-b668-6df99eedcb45 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.372081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1147/2296] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:52.376102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.393927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de4f8-bbe3-4304-a873-83c6a1dee192 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.419965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.420677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.421053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.421978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] PATH_INFO: `/v3/projects/9128d708b9c645cf914914a2378ff6ed/users/d2f10d8db93a411d92b1e92fbd33a141/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.447355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9128d708b9c645cf914914a2378ff6ed, user_id=d2f10d8db93a411d92b1e92fbd33a141, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.453393 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.477682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1150/2297] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:52 2022] PUT /identity/v3/projects/9128d708b9c645cf914914a2378ff6ed/users/d2f10d8db93a411d92b1e92fbd33a141/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:52.491085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb2431-59f2-4599-9819-390fda8fcbc0 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.491653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb2431-59f2-4599-9819-390fda8fcbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.492135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb2431-59f2-4599-9819-390fda8fcbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.539616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-affb2431-59f2-4599-9819-390fda8fcbc0 None None] MFA Rules not processed for user `d2f10d8db93a411d92b1e92fbd33a141`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:52.540851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-affb2431-59f2-4599-9819-390fda8fcbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:52.655740 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:52.693738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1148/2298] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:52.701678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.703675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:52.811113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9663ff-36e6-4b26-bbbd-bc27fd8625b2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:52.812573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.813026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.813134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.828547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:52.828795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:52.828903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:52.829159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:52.842132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:52.844021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:52.860597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] MFA Rules not processed for user `0c0cd616a3794428809f28dfbc86e7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:52.862316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:52.873233 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1151/2299] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:52.981473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1149/2300] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:39:52.982849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:52.984561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:53.024517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:53.106686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cff227-cd4d-410a-868e-d0288f9d93c0 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.115994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.117422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.117904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.118944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.120085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.122717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.156354 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1152/2301] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:39:52 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:39:53.161703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:53.178546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ee53b-5131-4795-bf04-37c3f2f55be7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.269911 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.271260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.272909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.273330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.276212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.279412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.323082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1150/2302] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:39:53 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:39:53.325132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:53.348882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8eeae-2a6c-47b1-95b2-621e3361e20d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.358814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.359537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.359919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.360300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.361068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.363615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.381709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1153/2303] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:53.385834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:53.399346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059fd5-3cd3-4134-bd1c-5037f53ab839 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.418115 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.418863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.419211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.419639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] PATH_INFO: `/v3/projects/2d05a3d521944c808d497e4d4b61411f/users/0c951148f20c4c4bb8d0c8284de55baa/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.450417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d05a3d521944c808d497e4d4b61411f, user_id=0c951148f20c4c4bb8d0c8284de55baa, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.456567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.477189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1151/2304] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:53 2022] PUT /identity/v3/projects/2d05a3d521944c808d497e4d4b61411f/users/0c951148f20c4c4bb8d0c8284de55baa/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:53.481524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:53.514005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd5ff-1f3a-4944-ab43-a14581ef773c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.601597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.602403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.602821 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.603270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.604064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.605926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b54e-339f-4c01-b709-42c16a789d4d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.619951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1154/2305] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:39:53.623820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:53.637323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d946a4-1918-470e-a800-5f100c704020 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:53.645501 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:53.646903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:53.648572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:53.648572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] PATH_INFO: `/v3/projects/2d05a3d521944c808d497e4d4b61411f/users/0c951148f20c4c4bb8d0c8284de55baa/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:53.652393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d05a3d521944c808d497e4d4b61411f, user_id=0c951148f20c4c4bb8d0c8284de55baa, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:53.656315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff07c9-f11f-4536-bce8-52cda5157bbd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:53.671428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1152/2306] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:39:53 2022] PUT /identity/v3/projects/2d05a3d521944c808d497e4d4b61411f/users/0c951148f20c4c4bb8d0c8284de55baa/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:39:54.461148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ecd6-ccf1-4915-bb9c-ab98da087884 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:54.461437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ecd6-ccf1-4915-bb9c-ab98da087884 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:54.461720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ecd6-ccf1-4915-bb9c-ab98da087884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:54.496742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4cd8ecd6-ccf1-4915-bb9c-ab98da087884 None None] MFA Rules not processed for user `d2f10d8db93a411d92b1e92fbd33a141`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:54.498204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4cd8ecd6-ccf1-4915-bb9c-ab98da087884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:54.604055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1155/2307] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:39:57.750062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:57.750969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:57.751446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:57.784566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] MFA Rules not processed for user `ff886c5f2516476e85e9e805aa19fece`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:39:57.786446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:57.846494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:57.909716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1153/2308] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:39:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:40:01.235772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbd25-900e-42a8-881c-326c86b21276 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:01.236316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbd25-900e-42a8-881c-326c86b21276 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:01.236993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bbd25-900e-42a8-881c-326c86b21276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:01.266741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c76bbd25-900e-42a8-881c-326c86b21276 None None] MFA Rules not processed for user `0c951148f20c4c4bb8d0c8284de55baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:01.268228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c76bbd25-900e-42a8-881c-326c86b21276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:01.290026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:01.350111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1156/2309] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:01.362803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:01.364283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:01.469173 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e4008-1f1f-4276-91fe-50bb6f8396ed None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:01.473879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:01.474824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:01.475367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:01.475854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:01.557132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:01.558934 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:01.586156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1154/2310] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:40:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:40:01.737945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2bf7-a19d-4d64-9b8f-2f87649bdd26 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:01.738377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2bf7-a19d-4d64-9b8f-2f87649bdd26 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:01.738804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2bf7-a19d-4d64-9b8f-2f87649bdd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:01.760473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-629a2bf7-a19d-4d64-9b8f-2f87649bdd26 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:01.762123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-629a2bf7-a19d-4d64-9b8f-2f87649bdd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:01.827925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1157/2311] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:40:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:01.881497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:01.883019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:01.962820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86335a0-37d0-4720-9d15-2c1d74e942c0 service glance] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:01.970819 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:01.971704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:01.972310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:01.972680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:01.982093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:01.983791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a6484-95fd-47f9-835d-3b40d0d92755 service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:01.993886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1155/2312] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:11.821086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=44 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:11.993936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:12.082924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:12.083213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:12.083512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:12.102893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] MFA Rules not processed for user `0c0cd616a3794428809f28dfbc86e7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:12.104107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:12.110967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1158/2313] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:15.746306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035806-75cf-4acb-80f7-5b1c93877e92 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:15.746793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035806-75cf-4acb-80f7-5b1c93877e92 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:15.746885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79035806-75cf-4acb-80f7-5b1c93877e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:15.766497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-79035806-75cf-4acb-80f7-5b1c93877e92 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:15.767751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-79035806-75cf-4acb-80f7-5b1c93877e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:15.775440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1156/2314] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:40:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:15.788245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:15.801382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751106-6f84-4844-983e-f64e68b16329 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:15.807592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:15.808770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:15.809059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:15.809342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:15.823899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:15.826690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:15.836929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1159/2315] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:16.778907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d595-09cb-4862-a89c-25c57a86c2ec None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:16.779420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d595-09cb-4862-a89c-25c57a86c2ec None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:16.779420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d595-09cb-4862-a89c-25c57a86c2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:16.804587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e4a2d595-09cb-4862-a89c-25c57a86c2ec None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:16.806394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e4a2d595-09cb-4862-a89c-25c57a86c2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:16.828983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1157/2316] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:40:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:16.860134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:16.879465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd57a-1242-4690-b9c7-29fca346013f service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:16.885454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:16.886400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:16.886777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:16.887275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:16.897170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:16.898829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:16.908311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1160/2317] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:17.642078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2c56b-3fac-4ebc-8e83-d22ba6e22637 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:17.642390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2c56b-3fac-4ebc-8e83-d22ba6e22637 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:17.642745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2c56b-3fac-4ebc-8e83-d22ba6e22637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:17.663542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-abe2c56b-3fac-4ebc-8e83-d22ba6e22637 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:17.664796 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-abe2c56b-3fac-4ebc-8e83-d22ba6e22637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:17.672286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1158/2318] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:40:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:17.695388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:17.707492 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8761-c5a1-4ed4-85a4-40609b0e0c3d service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:17.713010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:17.713618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:17.714080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:17.714509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:17.723832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:17.725686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:17.734574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1161/2319] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:18.047577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93b3d-6a78-4517-81a2-7fbbff5d4efd None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:18.048170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93b3d-6a78-4517-81a2-7fbbff5d4efd None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:18.048170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b93b3d-6a78-4517-81a2-7fbbff5d4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:18.068544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f6b93b3d-6a78-4517-81a2-7fbbff5d4efd None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:18.069907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f6b93b3d-6a78-4517-81a2-7fbbff5d4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:18.077420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1159/2320] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:40:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:18.091391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:18.102619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87481ecd-0c39-4dca-9127-cb6374d3f994 service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:18.108139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:18.108754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:18.109239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:18.109920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:18.120444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:18.122832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:18.133416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1162/2321] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:18.632197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:18.632844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:18.633285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:18.658001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:18.659463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:18.673421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1160/2322] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:40:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:18.686922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:18.708089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cd676-f402-4cdf-9b17-c89d509bb15e service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:18.729490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:18.730110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:18.730390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:18.730727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:18.759597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:18.763275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b08623-f640-4b96-b4ef-36d42a367777 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:18.773943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1163/2323] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:21.140888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:21.218415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817830fd-574c-4668-8975-dab3d2ada749 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:21.241383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:21.242146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:21.242495 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:21.246965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] PATH_INFO: `/v3/users/d2f10d8db93a411d92b1e92fbd33a141` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:21.248319 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f10d8db93a411d92b1e92fbd33a141)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:21.250321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:21.323059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f10d8db93a411d92b1e92fbd33a141'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.324213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f10d8db93a411d92b1e92fbd33a141'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.324950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f10d8db93a411d92b1e92fbd33a141'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.335821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f10d8db93a411d92b1e92fbd33a141'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.357899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f10d8db93a411d92b1e92fbd33a141'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.374581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1161/2324] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:40:21 2022] DELETE /identity/v3/users/d2f10d8db93a411d92b1e92fbd33a141 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:40:21.379020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:21.379571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:21.379571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:21.400152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:21.401466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:21.469001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1164/2325] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:21.479778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:21.481455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:21.544028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:21.550757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e69ea-97a3-4534-9bc3-80f8a622c973 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:21.556181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:21.557446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:21.557734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:21.558132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:21.569633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:21.571132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:21.581376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1162/2326] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:40:21.776253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:21.777696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:21.792300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb3dd1-5901-46ad-a82b-a031c59095e8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:21.810364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:21.810992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:21.811349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:21.811698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] PATH_INFO: `/v3/projects/9128d708b9c645cf914914a2378ff6ed` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:21.813137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9128d708b9c645cf914914a2378ff6ed)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:21.815617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:21.880903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9128d708b9c645cf914914a2378ff6ed', 'request_id': 'req-e6e7af78-d190-411e-b7be-38b577d140b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.887653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9128d708b9c645cf914914a2378ff6ed', 'request_id': 'req-e6e7af78-d190-411e-b7be-38b577d140b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.894542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9128d708b9c645cf914914a2378ff6ed', 'request_id': 'req-e6e7af78-d190-411e-b7be-38b577d140b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.917929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] The token cache is being invalidate because project 9128d708b9c645cf914914a2378ff6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:40:21.918280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.919122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e6e7af78-d190-411e-b7be-38b577d140b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:21.921109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1165/2327] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:40:21 2022] DELETE /identity/v3/projects/9128d708b9c645cf914914a2378ff6ed => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:40:24.040593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:24.042520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:24.115207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a3fe1-6bca-4925-a9bd-a7ab90b66f1c service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:24.183933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:24.184576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:24.184857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:24.185136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:24.187013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:24.199902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:24.201523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:24.212586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1163/2328] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:40:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:40:30.511318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5789e9-13f4-447c-b275-e1359b36181e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:30.511798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5789e9-13f4-447c-b275-e1359b36181e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:30.511889 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5789e9-13f4-447c-b275-e1359b36181e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:30.531667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2a5789e9-13f4-447c-b275-e1359b36181e None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:30.532881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2a5789e9-13f4-447c-b275-e1359b36181e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:30.539325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:30.596666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1166/2329] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:40:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:30.611176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:30.613441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:30.684232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28124411-b9a9-4b43-9faf-9b68ed6e4501 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:30.689220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:30.690295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:30.690295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:30.690506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:30.698842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:30.700370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:30.710982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1164/2330] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:40.588608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=20 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:40.705574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:46.133084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:46.133592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:46.133592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:46.153110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:46.154308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:46.161586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1167/2331] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:40:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:46.174526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:46.185955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36788f50-37a6-4e28-b80c-2c7a22cefdd7 service swift] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:46.191957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:46.192556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:46.192840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:46.193110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:46.201428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:46.203025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:46.212039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1165/2332] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:40:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:40:46.417679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:46.419181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:46.487157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b907-54bb-4481-b52d-3a14c8001a25 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:46.492551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:46.493222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:46.493452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:46.493921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:46.495768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:46.563452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:46.564884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:46.574910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1168/2333] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:40:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:40:49.020429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:49.022331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:49.143018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524c305-37ec-4947-96a3-2e432af460be service swift] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:49.148660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:49.149293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:49.149572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:49.149847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] PATH_INFO: `/v3/users/ff886c5f2516476e85e9e805aa19fece` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:49.151133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff886c5f2516476e85e9e805aa19fece)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:49.156672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:49.237045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff886c5f2516476e85e9e805aa19fece'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:49.237946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff886c5f2516476e85e9e805aa19fece'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:49.238938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff886c5f2516476e85e9e805aa19fece'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:49.256599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff886c5f2516476e85e9e805aa19fece'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:49.282631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a7a52fdb-672a-48d8-a22e-6250bda2d2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff886c5f2516476e85e9e805aa19fece'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:49.328822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1166/2334] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:40:49 2022] DELETE /identity/v3/users/ff886c5f2516476e85e9e805aa19fece => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:40:49.729549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:49.744947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f437-90ca-49c3-8b84-d7e130fffc5e service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:49.910559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:49.911247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:49.911528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:49.911801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] PATH_INFO: `/v3/projects/df1ee75d012f4378a607020154eaf0b4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:49.913177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1ee75d012f4378a607020154eaf0b4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:49.915597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.010292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:50.013634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1ee75d012f4378a607020154eaf0b4', 'request_id': 'req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:50.020054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1ee75d012f4378a607020154eaf0b4', 'request_id': 'req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:50.028211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1ee75d012f4378a607020154eaf0b4', 'request_id': 'req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:50.045918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] The token cache is being invalidate because project df1ee75d012f4378a607020154eaf0b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:40:50.046632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:50.047884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8fc53e07-e1f0-4a9c-8965-2dcb7fc5efed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:40:50.050730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1169/2335] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:40:49 2022] DELETE /identity/v3/projects/df1ee75d012f4378a607020154eaf0b4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:40:50.054802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.055286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.055565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.077396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:50.078781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:50.150320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1167/2336] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:50.158385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.158694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.158991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.187995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:50.189468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:50.199696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1170/2337] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:40:50.203981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.215825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78435e-70fb-45f8-b703-3dde3d55e395 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.223626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.224611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.224933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.225655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.226656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:40:50.227648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.230422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.233321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:40:50.250236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1168/2338] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:40:50.254081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.264771 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef10604-7d8a-433b-b270-d7e8e41c198e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.274057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.274835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.275220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.276053 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.276430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.278665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.310847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1171/2339] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:40:50 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:40:50.314463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.325691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7ed12-30f1-4678-93c0-a425e1980622 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.397278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.398003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.398294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.398809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.399283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.401666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.429897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1169/2340] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:40:50 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:40:50.434578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.445929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4ad0-0003-4fe2-b965-3062722eebdb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.451167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.451861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.452211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.452561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.453312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.455093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.466731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1172/2341] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:40:50.470782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.481967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff66e64-8e1d-43f6-a9b8-41c2e237a599 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.486676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.487300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.487582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.487856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.488699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.490299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bdaf-a8c9-4d56-820f-e2102581f9fd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.502685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1170/2342] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:40:50.506106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.517141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7cf13-c34a-498d-b25e-86bad77e2d93 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.523022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.523803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.524089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.524367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] PATH_INFO: `/v3/projects/ad9c62324f054c6c9e653e9192eb367e/users/bcff320bd53b48aea4a448b5cd46aa73/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.541997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9c62324f054c6c9e653e9192eb367e, user_id=bcff320bd53b48aea4a448b5cd46aa73, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.544700 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.561933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1173/2343] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:40:50 2022] PUT /identity/v3/projects/ad9c62324f054c6c9e653e9192eb367e/users/bcff320bd53b48aea4a448b5cd46aa73/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:40:50.573593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.573982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.574486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.604655 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] MFA Rules not processed for user `bcff320bd53b48aea4a448b5cd46aa73`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:40:50.606034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:50.673077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1171/2344] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:40:50.684129 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:40:50.685900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:40:50.749772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc74e9-89fd-4043-8742-c05e33511668 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:40:50.754477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:40:50.755054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:40:50.755337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:40:50.755638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:40:50.764387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:40:50.765828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b3b4a-e427-4548-8758-8ff1c530c513 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:40:50.776002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1174/2345] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:40:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:41:00.670155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:00.771860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:01.430344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:01.495506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8242-6d04-429f-87f8-42b38e3628da None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:01.500113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:01.500746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:01.501030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:01.501305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] PATH_INFO: `/v3/users/bcff320bd53b48aea4a448b5cd46aa73` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:01.502462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcff320bd53b48aea4a448b5cd46aa73)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:01.505535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:01.554007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcff320bd53b48aea4a448b5cd46aa73'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:01.554690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcff320bd53b48aea4a448b5cd46aa73'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:01.555770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcff320bd53b48aea4a448b5cd46aa73'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:01.563696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcff320bd53b48aea4a448b5cd46aa73'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:01.577050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcff320bd53b48aea4a448b5cd46aa73'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:01.589804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1172/2346] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:41:01 2022] DELETE /identity/v3/users/bcff320bd53b48aea4a448b5cd46aa73 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:41:01.594357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:01.594760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:01.595118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:01.615985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:01.617206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:01.684207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1175/2347] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:01.695030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:01.696521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:01.767388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab138abf-1de3-4160-8c5e-ffb4a0de779f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:01.775341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:01.775975 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:01.776253 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:01.776525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:01.787357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:01.788832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290f6c1-db27-419f-bd06-65bd6396e5f5 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:01.801053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1173/2348] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:41:02.172956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.174543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:02.199430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7685d-0aac-4216-ac48-e942baa50252 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.204614 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.205304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.205556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.205836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] PATH_INFO: `/v3/projects/ad9c62324f054c6c9e653e9192eb367e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.208256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9c62324f054c6c9e653e9192eb367e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.210192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.263279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9c62324f054c6c9e653e9192eb367e', 'request_id': 'req-72ead245-e8b1-4cf6-b69f-a7606b015183'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:02.270789 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9c62324f054c6c9e653e9192eb367e', 'request_id': 'req-72ead245-e8b1-4cf6-b69f-a7606b015183'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:02.277697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9c62324f054c6c9e653e9192eb367e', 'request_id': 'req-72ead245-e8b1-4cf6-b69f-a7606b015183'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:02.292668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] The token cache is being invalidate because project ad9c62324f054c6c9e653e9192eb367e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:41:02.292906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:02.293523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72ead245-e8b1-4cf6-b69f-a7606b015183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:41:02.295286 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1176/2349] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:41:02 2022] DELETE /identity/v3/projects/ad9c62324f054c6c9e653e9192eb367e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:41:02.301338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.301749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.302137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.325309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:02.326592 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:02.344382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:02.391814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1174/2350] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:02.399412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.399710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.399986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.423930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:02.425713 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:02.434549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1177/2351] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:02.439003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.450394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afd9e4-89c3-4f81-acc0-67c82e01f107 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.455026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.455627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.455926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.456710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.457646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:41:02.458032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.459658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.471630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1175/2352] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:41:02.476964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.486609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68d69d-c4cd-4f72-be24-71f885ebfdbd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.491438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.492038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.492318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.492590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.493226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.495173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.523725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1178/2353] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:41:02 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:41:02.527868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.539270 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782cea2-7380-4a87-9b8b-c0e88d0f8127 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.610792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.611408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.611692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.611974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.612673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.614959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.638543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1176/2354] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:41:02 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:41:02.642697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.653663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d9b6-456e-4840-b2fd-b01b87ced53d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.658748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.659394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.660073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.660073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.660814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.662434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.673267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1179/2355] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:41:02.677273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.687126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c382e7-108f-4bc1-8e7d-d7622c16f281 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.692337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.692929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.693205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.693475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.694107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.695732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997f0d7-5c70-4944-b146-0b95e4827d0c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.709897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1177/2356] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:41:02.711068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.721414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5a46-6a31-4e7d-b439-c20061836362 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.726833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.727784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.727784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.728033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] PATH_INFO: `/v3/projects/0b578ad7d48e406fabfc32450de322d1/users/685fc311a8c14573b19719fb093e3ee2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.742934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b578ad7d48e406fabfc32450de322d1, user_id=685fc311a8c14573b19719fb093e3ee2, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.745759 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.759076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1180/2357] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:41:02 2022] PUT /identity/v3/projects/0b578ad7d48e406fabfc32450de322d1/users/685fc311a8c14573b19719fb093e3ee2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:41:02.764341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff642de-2d66-43d0-b260-f3ce347e5413 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.764629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff642de-2d66-43d0-b260-f3ce347e5413 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.764903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff642de-2d66-43d0-b260-f3ce347e5413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.785879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5ff642de-2d66-43d0-b260-f3ce347e5413 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:02.787411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5ff642de-2d66-43d0-b260-f3ce347e5413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:02.852923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1178/2358] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:02.862522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:02.866044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:02.890536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:02.932764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06956c-f55f-4382-bf4b-b0198ab44c20 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:02.937093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:02.937840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:02.937973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:02.938257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:02.950221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:02.951800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868071ca-cb80-4c5d-a043-15fb4dfbd664 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:02.962307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1181/2359] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:41:03.236614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:03.237163 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:03.237445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:03.279999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] MFA Rules not processed for user `1541e1614d664891b1952379b003ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:03.281612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:03.335584 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:03.375865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1179/2360] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:03.898791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300995c4-56e0-4035-9626-db2f08363ea4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:03.899666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300995c4-56e0-4035-9626-db2f08363ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:03.900089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300995c4-56e0-4035-9626-db2f08363ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:03.935072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-300995c4-56e0-4035-9626-db2f08363ea4 None None] MFA Rules not processed for user `685fc311a8c14573b19719fb093e3ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:03.936556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-300995c4-56e0-4035-9626-db2f08363ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:04.023119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1182/2361] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:04.034897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:41:04.036569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:04.098701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55021a08-88c5-47cc-a9e5-80149d29dbd2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:41:04.102622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:41:04.103332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:04.103607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:04.103874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:04.113620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:41:04.115127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15358559-7cd5-44df-83b8-89b9aaa30b47 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:41:04.124487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1180/2362] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:41:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:41:05.535506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f8a-075a-43c4-aa72-36fdc4ed37aa None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:05.536040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f8a-075a-43c4-aa72-36fdc4ed37aa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:05.536040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f8a-075a-43c4-aa72-36fdc4ed37aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:05.555446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-54766f8a-075a-43c4-aa72-36fdc4ed37aa None None] MFA Rules not processed for user `685fc311a8c14573b19719fb093e3ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:05.556671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-54766f8a-075a-43c4-aa72-36fdc4ed37aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:05.563844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1183/2363] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:14.119965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=38 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:15.549261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=52 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:19.635114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bda4b-2a9d-4722-9958-aa17e7514255 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:19.635724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bda4b-2a9d-4722-9958-aa17e7514255 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:19.635724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bda4b-2a9d-4722-9958-aa17e7514255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:19.660630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-385bda4b-2a9d-4722-9958-aa17e7514255 None None] MFA Rules not processed for user `4dd615d7af7741b08cc44b6c983fae49`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:19.662095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-385bda4b-2a9d-4722-9958-aa17e7514255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:19.748867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1181/2364] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:29.732024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=25 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:41:38.083734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:41:38.084310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:41:38.084310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:41:38.105784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] MFA Rules not processed for user `f28236459bb9492eaa35690f121d4e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:41:38.107054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:41:38.161475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1184/2365] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:41:48.154586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:13.448596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:13.449174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:13.449174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:13.469921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] MFA Rules not processed for user `685fc311a8c14573b19719fb093e3ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:13.471067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:13.478008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1182/2366] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:13.516659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:13.518052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:13.576949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e519-bdde-45c4-9649-751b16e696d9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:13.580830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:13.581476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:13.581760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:13.582038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:13.583829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:13.654252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:13.655737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:13.667285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1185/2367] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:17.547669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:17.549233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:17.561854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e52926-660b-4a53-bdd8-e57da8eaf10a None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:17.565555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:17.566155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:17.566431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:17.566733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:17.568461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:17.582123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:17.583530 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:17.594390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1183/2368] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:23.663709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=54 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:27.589698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=9 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:31.330603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:31.331997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:31.362402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71151afd-a7dd-4502-acc1-ee8470dde9d9 service glance] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:31.368033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:31.368968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:31.369252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:31.369522 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:31.371422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:31.384194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:31.384773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:31.392907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:31.394383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:31.404638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1186/2369] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:31.409615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eede3df-77f5-40b5-bb27-e974ab3f19bd service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:31.417320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:31.418192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:31.418706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:31.419204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:31.421958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:31.434186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:31.435902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4e94-9d07-4d86-b52c-a2f2173cc456 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:31.446946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1184/2370] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:37.767126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:37.775002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502500-99f1-49be-8c66-809f86cbf2cb service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:37.780788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:37.781736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:37.782014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:37.782284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:37.784398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:37.796900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:37.798366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:37.808331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1187/2371] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:40.294181 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:40.295229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:40.295760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:40.339269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] MFA Rules not processed for user `1541e1614d664891b1952379b003ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:40.340512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:40.350130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1185/2372] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:40.849375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:40.862312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75e123-c93c-4f2b-88a9-48e740649db8 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:40.866332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:40.866992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:40.867275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:40.867676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:40.869787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:40.884527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:40.886610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:40.919564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1188/2373] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:41.356458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:41.362083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.402384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:41.405144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.508882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99af594-44e6-4aab-b274-eb371d374a0e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:41.517586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:41.518240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:41.519169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:41.519464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:41.521196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.535811 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:41.537442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:41.543048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8608f-d497-4c4a-b9fd-c1710903cba2 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:41.548118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1186/2374] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:41.548950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:41.550420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.551592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:41.552539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:41.553081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:41.553669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:41.557836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.567580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1503e2-2e68-4e1e-91b7-0636698d0dea service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:41.572639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:41.573276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:41.573721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:41.574182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] PATH_INFO: `/v3/users/4dd615d7af7741b08cc44b6c983fae49` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:41.575128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:41.576405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd615d7af7741b08cc44b6c983fae49)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:41.577455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:41.578753 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:41.588143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1189/2375] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:41.631023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd615d7af7741b08cc44b6c983fae49'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:41.631705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd615d7af7741b08cc44b6c983fae49'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:41.632438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd615d7af7741b08cc44b6c983fae49'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:41.640912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd615d7af7741b08cc44b6c983fae49'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:41.656531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd615d7af7741b08cc44b6c983fae49'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:41.668865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1187/2376] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:42:41 2022] DELETE /identity/v3/users/4dd615d7af7741b08cc44b6c983fae49 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:42:41.788195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:41.803300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d7d5-ed72-4908-b28b-931830586b0a service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:41.854962 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:41.856779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.905624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a52376-2944-4964-a47d-990b758a666d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:41.906829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a52376-2944-4964-a47d-990b758a666d service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:41.907102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a52376-2944-4964-a47d-990b758a666d service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:41.907383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a52376-2944-4964-a47d-990b758a666d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:41.909884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37a52376-2944-4964-a47d-990b758a666d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:41.929602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:41.957728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d3c8-5179-4d29-9cbc-88553519b895 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:41.963124 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:41.963868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:41.964156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:41.964434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] PATH_INFO: `/v3/projects/3d204b26857042e397b91be285fa3ed9` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:41.965579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d204b26857042e397b91be285fa3ed9)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:41.965865 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:41.967961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:41.979367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a52376-2944-4964-a47d-990b758a666d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:41.981092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a52376-2944-4964-a47d-990b758a666d service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:41.991555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1190/2377] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:42.024439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d204b26857042e397b91be285fa3ed9', 'request_id': 'req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:42.031345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d204b26857042e397b91be285fa3ed9', 'request_id': 'req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:42.038866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d204b26857042e397b91be285fa3ed9', 'request_id': 'req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:42.051947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] The token cache is being invalidate because project 3d204b26857042e397b91be285fa3ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:42:42.052240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:42.053026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-2bc9f489-0c0d-4d76-a62f-9e5d1822012b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:42.054726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1188/2378] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:42:41 2022] DELETE /identity/v3/projects/3d204b26857042e397b91be285fa3ed9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:42:42.059922 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.060241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.060605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.080569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:42.081775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:42.145040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1191/2379] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:42:42.152651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.152939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.153367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.175515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:42.176913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:42.186221 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1189/2380] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:42.190110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.202719 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d82ad-fe6c-47cb-bc84-9b8425327e09 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.208110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.208786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.209137 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.209480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.210202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:42:42.210576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.212265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.230850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1192/2381] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:42:42.234523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.245879 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153647-aaed-43cd-8910-52fc72ca3ef8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.251869 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.252787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.253054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.253692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.254600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.257015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.310496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1190/2382] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:42:42 2022] POST /identity/v3/projects => generated 389 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:42:42.319574 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.337342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadac16-f709-475d-b3bc-7cfa7a420976 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.408782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.409389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.409670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.409973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.410598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.412654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.434347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1193/2383] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:42:42 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:42:42.439367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.449475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd69291-cd88-42a4-b3b7-7362e03606e6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.454499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.455117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.455398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.455670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.456315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.458197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.468352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1191/2384] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:42:42.472316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.483011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111ae0-146a-4467-ac3f-9b9dbfeca8ff admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.489553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.490177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.490461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.490767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.491421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.493058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873b714-a984-41dc-b97c-8b029851c378 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.505177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1194/2385] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:42:42.508990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.519196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef72d-a26e-4c76-9c72-fd43fc91d9cd admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.524328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.524986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.525268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.525544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] PATH_INFO: `/v3/projects/9b9728b1ee454013a3968c27080cd98c/users/e02f332ba23a4a3cacb99f5c69a5b524/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.541817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9728b1ee454013a3968c27080cd98c, user_id=e02f332ba23a4a3cacb99f5c69a5b524, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.545110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.562252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1192/2386] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:42:42 2022] PUT /identity/v3/projects/9b9728b1ee454013a3968c27080cd98c/users/e02f332ba23a4a3cacb99f5c69a5b524/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:42:42.567261 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.567551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.567824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.586491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:42.587726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:42.635867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:42.649413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1195/2387] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:42.659791 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:42.661499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:42.724123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c249510-392a-4baa-8965-03fe38246dda admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:42.728315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:42.728923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:42.729236 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:42.729473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:42.740018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:42.741465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7798df-a491-495d-a098-5ba7eecc51c3 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:42.751097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1193/2388] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:46.966416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:46.987697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837044d7-a02c-4fd1-a1b8-1a07e13c3006 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:46.993831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:46.994603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:46.995008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:46.995389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:46.997915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:47.009394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:47.011618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:47.024240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1196/2389] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:47.507886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:47.508793 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:47.509415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:47.534087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:47.535440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:47.541057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:47.604437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1194/2390] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:47.675540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:47.677069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:47.797085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c74943-7d2b-48e5-8037-ccdb095c3769 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:47.811153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:47.812118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:47.815750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:47.816547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:47.831739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:47.833435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bfe67-6d63-478e-b641-af85ebe1859a service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:47.843222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1197/2391] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:42:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:42:48.736428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:48.737831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:48.908579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dab81-54c1-4da0-a665-ee09b131573e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:48.915942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:48.916814 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:48.917157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:48.917494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:48.921146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:48.943258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:48.946440 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:48.970513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1195/2392] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:49.163205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375469e-dd47-40a4-a413-71ab6c9f7284 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:49.168046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375469e-dd47-40a4-a413-71ab6c9f7284 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:49.169183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375469e-dd47-40a4-a413-71ab6c9f7284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:49.271089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5375469e-dd47-40a4-a413-71ab6c9f7284 None None] MFA Rules not processed for user `e02f332ba23a4a3cacb99f5c69a5b524`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:49.272412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5375469e-dd47-40a4-a413-71ab6c9f7284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:49.392694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1198/2393] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:49.407634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:49.422946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75a9c-11d6-4e13-a49b-8bddf2b2c36a service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:49.429756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:49.431205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:49.431680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:49.432166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:49.446133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:49.448094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:49.463487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1196/2394] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:51.540097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:51.540594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:51.540808 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:51.561766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] MFA Rules not processed for user `e02f332ba23a4a3cacb99f5c69a5b524`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:51.563326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:51.576505 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1199/2395] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:54.049690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:54.051383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:54.159533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a74ab7-8996-4e2a-921b-7e5a7a04b9e8 service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:54.172806 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:54.173708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:54.173993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:54.174266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:54.176411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:54.196749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:54.198600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:54.211471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1197/2396] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:59.733693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:59.735277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:59.750108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dedef-4a1b-41d5-9b15-b119c3e9f675 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:59.754980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:59.755742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:59.756090 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:59.756433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] PATH_INFO: `/v3/users/0c0cd616a3794428809f28dfbc86e7b9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:59.757673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0cd616a3794428809f28dfbc86e7b9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:59.759722 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:59.817743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0cd616a3794428809f28dfbc86e7b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:59.819746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:59.820524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0cd616a3794428809f28dfbc86e7b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:59.821218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:59.821939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:59.829336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0cd616a3794428809f28dfbc86e7b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:59.830243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0cd616a3794428809f28dfbc86e7b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:59.838110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0cd616a3794428809f28dfbc86e7b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:42:59.838674 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c4a3-9928-4add-b165-97f6382bb393 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:59.845587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:59.846279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:59.846651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:59.847040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:59.848949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:59.860660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1200/2397] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:42:59 2022] DELETE /identity/v3/users/0c0cd616a3794428809f28dfbc86e7b9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:42:59.863202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:59.864919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:59.876980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1198/2398] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:00.076119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:00.077589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:00.179592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbcaab-4c1c-44a7-9e37-1ceac9fc34d1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:00.185844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:00.186631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:00.187372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:00.187752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] PATH_INFO: `/v3/projects/0159b61c064b4eca9fcb9ed3c1fff2b5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:00.189057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0159b61c064b4eca9fcb9ed3c1fff2b5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:00.191081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:00.260114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0159b61c064b4eca9fcb9ed3c1fff2b5', 'request_id': 'req-02c8e30f-228f-4796-aa9a-40a7d18377b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.266818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0159b61c064b4eca9fcb9ed3c1fff2b5', 'request_id': 'req-02c8e30f-228f-4796-aa9a-40a7d18377b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.273559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0159b61c064b4eca9fcb9ed3c1fff2b5', 'request_id': 'req-02c8e30f-228f-4796-aa9a-40a7d18377b9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.294340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] The token cache is being invalidate because project 0159b61c064b4eca9fcb9ed3c1fff2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:00.295016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.295494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.300595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1201/2399] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:00 2022] DELETE /identity/v3/projects/0159b61c064b4eca9fcb9ed3c1fff2b5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:00.301703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:00.308098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:00.332114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f979064-42c8-47c6-8489-ff96aef02251 service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:00.402230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:00.403754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:00.404251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:00.404744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] PATH_INFO: `/v3/users/0c951148f20c4c4bb8d0c8284de55baa` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:00.406431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c951148f20c4c4bb8d0c8284de55baa)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:00.408771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:00.482359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c951148f20c4c4bb8d0c8284de55baa'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.483413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c951148f20c4c4bb8d0c8284de55baa'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.484194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c951148f20c4c4bb8d0c8284de55baa'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.500742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c951148f20c4c4bb8d0c8284de55baa'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.520089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-92c7ab66-adf2-461f-b671-6d1431652337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c951148f20c4c4bb8d0c8284de55baa'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:00.541222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1199/2400] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:00 2022] DELETE /identity/v3/users/0c951148f20c4c4bb8d0c8284de55baa => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:01.001850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.003426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:01.022219 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e30f-228f-4796-aa9a-40a7d18377b9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.109044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:01.119893 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.122294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.122945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.123416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] PATH_INFO: `/v3/projects/2d05a3d521944c808d497e4d4b61411f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.124884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d05a3d521944c808d497e4d4b61411f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.127562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.176800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d05a3d521944c808d497e4d4b61411f', 'request_id': 'req-88efc3ff-9a24-416e-b83d-86d7d9effbc1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:01.187737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d05a3d521944c808d497e4d4b61411f', 'request_id': 'req-88efc3ff-9a24-416e-b83d-86d7d9effbc1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:01.197634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d05a3d521944c808d497e4d4b61411f', 'request_id': 'req-88efc3ff-9a24-416e-b83d-86d7d9effbc1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:01.210926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] The token cache is being invalidate because project 2d05a3d521944c808d497e4d4b61411f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:01.211222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:01.211847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-88efc3ff-9a24-416e-b83d-86d7d9effbc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:01.213819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1202/2401] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:01 2022] DELETE /identity/v3/projects/2d05a3d521944c808d497e4d4b61411f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:01.218897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.219410 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.219521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.239599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:01.240802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:01.301801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1200/2402] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:01.314730 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.315017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.315292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.334895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:01.336076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:01.344559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1203/2403] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:43:01.348804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.360191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c7d57-681a-4da6-bf50-933910d8fb13 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.364504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.365099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.365377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.365647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.366295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:43:01.366596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.368207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.379851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1201/2404] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:43:01.384010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.393502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa7b7a-7034-4b96-9eb9-b23d835908ae None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.398117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.398732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.399013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.399284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.399940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.402601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.433588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1204/2405] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:01 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:01.434720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.444746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817fb74-8c18-4f38-8c5b-7685c998328a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.452822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:01.502805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.503445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.503726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.504002 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.504643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.506583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.527550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1202/2406] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:01 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:01.531583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.541930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b585d3-5469-4f77-a873-c6a81359d3d4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.547351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.547796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.548072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.548344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.549000 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.550600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.561721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1205/2407] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:01.570734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.581300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374bcd-d41d-4746-a126-7d312fb099f4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.585925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.586526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.586835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.587129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.587757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.590043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca3842-2daa-454f-b2e0-78b032c874c3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.600843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1203/2408] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:01.606257 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.616553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896e3d3-4591-47d3-ac2d-93771b6701bb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.621230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.621950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.622230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.622504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] PATH_INFO: `/v3/projects/b496441965834f65a8d0813e8221c685/users/81be4c83d3b04624ae11bcbf37a44309/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.638494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b496441965834f65a8d0813e8221c685, user_id=81be4c83d3b04624ae11bcbf37a44309, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.641188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.654994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1206/2409] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:01 2022] PUT /identity/v3/projects/b496441965834f65a8d0813e8221c685/users/81be4c83d3b04624ae11bcbf37a44309/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:01.659974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.660282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.660562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.679665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:01.680862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:01.740360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1204/2410] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:01.749979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:01.751692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:01.806911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a9ad-ff60-4d96-b3cb-4096e4e75f45 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:01.812120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:01.812728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:01.813006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:01.813280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:01.822542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:01.824715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d8aef-ae3c-4820-8501-c7c625bbb08a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:01.834308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1207/2411] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:05.150837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:05.152447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:05.241310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a29752-07d7-475b-8d9f-aeadcbf03541 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:05.248097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:05.248724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:05.249017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:05.249282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:05.252038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:05.266552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:05.269283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:05.281567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1205/2412] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:06.465311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:06.465846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:06.465931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:06.494945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] MFA Rules not processed for user `81be4c83d3b04624ae11bcbf37a44309`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:06.496618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:06.509102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:06.628426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1208/2413] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:06.633020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:06.644742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e34f0-bfa3-42ab-9cda-ec78ba7aef39 service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:06.649534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:06.650137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:06.650422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:06.650721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:06.651434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:06.656398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:06.678595 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:06.695210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1206/2414] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:06 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:06.699986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:06.711385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca30b49-7f40-4372-97f7-3516dc26af19 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:06.787328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:06.788289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:06.788756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:06.789215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:06.790070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:06.792994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:06.830441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:06.836511 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1209/2415] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:06 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:06.849220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94bdaf-a715-4a4a-ad56-840eb34aae1d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:06.854268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:06.854841 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:06.855126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:06.856117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:06.856308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:06.857909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:06.873109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1207/2416] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:06.874553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:06.887681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e3319-32b8-4748-ac8c-d143988dd662 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:06.895360 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:06.896130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:06.896767 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:06.897267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] PATH_INFO: `/v3/projects/e232defe64dc4838abc09dbe8d1df3b3/users/c22dc9314f7648a3a7d25b12fc351a4f/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:06.919550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e232defe64dc4838abc09dbe8d1df3b3, user_id=c22dc9314f7648a3a7d25b12fc351a4f, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:06.922815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:06.945990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1210/2417] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:06 2022] PUT /identity/v3/projects/e232defe64dc4838abc09dbe8d1df3b3/users/c22dc9314f7648a3a7d25b12fc351a4f/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:06.949930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:06.960239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6cfb7-356e-4840-8253-039588ef0a11 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:07.054491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:07.055479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:07.055792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:07.056070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:07.056740 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:07.059384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f202f9-296d-4755-b4fc-71c3f34ec232 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:07.085763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1208/2418] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:07.086962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:07.102965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8aa2f-59b9-4569-8b24-6019e656ff2c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:07.113600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:07.114324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:07.114824 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:07.115752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] PATH_INFO: `/v3/projects/e232defe64dc4838abc09dbe8d1df3b3/users/c22dc9314f7648a3a7d25b12fc351a4f/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:07.117607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e232defe64dc4838abc09dbe8d1df3b3, user_id=c22dc9314f7648a3a7d25b12fc351a4f, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:07.122987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:07.153959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1211/2419] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:07 2022] PUT /identity/v3/projects/e232defe64dc4838abc09dbe8d1df3b3/users/c22dc9314f7648a3a7d25b12fc351a4f/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:12.012555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:12.013009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:12.013175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:12.064091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] MFA Rules not processed for user `c22dc9314f7648a3a7d25b12fc351a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:12.065365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:12.191700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1209/2420] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:12.210139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:12.395186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556951cd-3929-4ce5-b0c2-9ab369e87768 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:12.396473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:12.405436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:12.406262 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:12.406613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:12.407445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:12.497201 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:12.499019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:12.513855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1212/2421] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:21.117409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:21.174461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a5c2-c7bc-4c4c-a948-045676690669 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:21.178097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:21.178731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:21.179016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:21.179322 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:21.181271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:21.200602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:21.250388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:21.252130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:21.263062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1210/2422] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:22.507742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=26 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:23.745092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:23.747217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:23.774379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d10eb6-c5d3-4f42-ab6f-e336fedbdb77 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:23.780151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:23.780796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:23.781481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:23.781607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] PATH_INFO: `/v3/users/685fc311a8c14573b19719fb093e3ee2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:23.782697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685fc311a8c14573b19719fb093e3ee2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:23.784783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:23.851979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685fc311a8c14573b19719fb093e3ee2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:23.852803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685fc311a8c14573b19719fb093e3ee2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:23.861729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685fc311a8c14573b19719fb093e3ee2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:23.862494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685fc311a8c14573b19719fb093e3ee2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:23.871827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2be7ae08-85b6-44a2-996e-efd3a9ab8288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685fc311a8c14573b19719fb093e3ee2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:23.901895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1213/2423] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:23 2022] DELETE /identity/v3/users/685fc311a8c14573b19719fb093e3ee2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:24.102230 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.123741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1146932-67c1-40e9-bc0e-8011bc7f529e service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.236887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.237658 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.238010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.238351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] PATH_INFO: `/v3/projects/0b578ad7d48e406fabfc32450de322d1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.239720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b578ad7d48e406fabfc32450de322d1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.242984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.317105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b578ad7d48e406fabfc32450de322d1', 'request_id': 'req-d95de694-e861-4350-99a2-5c7e7c829cca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.326308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b578ad7d48e406fabfc32450de322d1', 'request_id': 'req-d95de694-e861-4350-99a2-5c7e7c829cca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.335656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b578ad7d48e406fabfc32450de322d1', 'request_id': 'req-d95de694-e861-4350-99a2-5c7e7c829cca'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.355210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] The token cache is being invalidate because project 0b578ad7d48e406fabfc32450de322d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:24.355706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.356435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d95de694-e861-4350-99a2-5c7e7c829cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.358024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1211/2424] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:24 2022] DELETE /identity/v3/projects/0b578ad7d48e406fabfc32450de322d1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:24.368299 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.370451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.370795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.397853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:24.399382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:24.481080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1214/2425] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:24.482807 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.483099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.483377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.505951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:24.507159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:24.515834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1212/2426] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:24.520165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.534772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1780584-3a27-4331-a253-ee6a59a2ac41 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.539326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.539957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.540237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.540514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.541176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:43:24.541483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.543202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.559762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1215/2427] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:43:24.560500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.579951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea97a3f-dee5-41a9-b857-efd3237a00a3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.585099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.585713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.585994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.586266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.587003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.589371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.646683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1213/2428] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:24 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:24.648370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.660906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fb925-5cdb-4666-af57-48381e5a233e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.759224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.759997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.760362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.760712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.762070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.765418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.779244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:24.804902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1216/2429] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:24 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:24.810107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.823856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.826124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:24.836074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfba844-c710-4179-a456-44b74c3ca9e2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.840641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4b25-2070-4dd8-bc0f-f27a8c6cf85a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.846409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.847581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.848039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.848170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.848603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] PATH_INFO: `/v3/users/f28236459bb9492eaa35690f121d4e3c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.848824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.849046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.849327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.850089 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.851050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f28236459bb9492eaa35690f121d4e3c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.853616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.854920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.867567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1214/2430] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:24.872656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.879559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:24.883673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392f85a-54ab-4123-83fd-3fdb4cd94b5e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.891764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.894081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.894081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.894081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.894081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.895948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:24.907326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1215/2431] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:24.911884 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:24.921374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f28236459bb9492eaa35690f121d4e3c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.922524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f28236459bb9492eaa35690f121d4e3c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.928143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6fae9-5748-43bd-afaf-6cede5feba93 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:24.933994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f28236459bb9492eaa35690f121d4e3c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.934926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f28236459bb9492eaa35690f121d4e3c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.937968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:24.938747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:24.939095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:24.940041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] PATH_INFO: `/v3/projects/3760b78ba8584e6c81c46d48113fcd45/users/90be4ab702e14f4387b0883dead72d9f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:24.945674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-daf0bc8a-c2c2-4c98-af48-e10919014ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f28236459bb9492eaa35690f121d4e3c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:24.965099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1217/2432] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:24 2022] DELETE /identity/v3/users/f28236459bb9492eaa35690f121d4e3c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:43:24.977348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3760b78ba8584e6c81c46d48113fcd45, user_id=90be4ab702e14f4387b0883dead72d9f, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:24.982186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.003514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1216/2433] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:24 2022] PUT /identity/v3/projects/3760b78ba8584e6c81c46d48113fcd45/users/90be4ab702e14f4387b0883dead72d9f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:25.008481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.008795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.009075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.035084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:25.036374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.123177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1218/2434] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:25.135448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.137217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.179857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.181352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.193677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfa7a4-f6e1-4d5a-bb82-a2c436f12ea4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.200236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.200887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.201165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.201450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] PATH_INFO: `/v3/projects/98f176b3cafe45d497dee2f1d031a4ec` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.202622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f176b3cafe45d497dee2f1d031a4ec)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.204753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.222667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95c9fb-ba50-4821-857d-d9143eef8ed6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.226831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.227461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.227747 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.228582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.239812 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.241360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262802a-d858-49f5-b1e5-ada3443eb682 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.253146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1217/2435] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:25.255633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f176b3cafe45d497dee2f1d031a4ec', 'request_id': 'req-e5da70ce-c36b-4c31-a0af-c1b6b586648d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:25.261743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=4,keystone:SELECT=80,keystone:DELETE=16 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:25.263699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f176b3cafe45d497dee2f1d031a4ec', 'request_id': 'req-e5da70ce-c36b-4c31-a0af-c1b6b586648d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:25.271972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f176b3cafe45d497dee2f1d031a4ec', 'request_id': 'req-e5da70ce-c36b-4c31-a0af-c1b6b586648d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:25.291663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] The token cache is being invalidate because project 98f176b3cafe45d497dee2f1d031a4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:25.292178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:25.293185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e5da70ce-c36b-4c31-a0af-c1b6b586648d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:25.295161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1219/2436] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:25 2022] DELETE /identity/v3/projects/98f176b3cafe45d497dee2f1d031a4ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:25.300860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.301154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.301432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.322343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:25.323531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.389882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1218/2437] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:25.397913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.398200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.398475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.422785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:25.424110 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.432773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1220/2438] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:43:25.437323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.447980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd0159-dfeb-4006-91db-94a8d553ab37 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.454564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.455204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.455485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.455761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.456415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:43:25.456716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.458311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.472956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1219/2439] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:43:25.477387 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.488195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7321e98-d4d6-483a-a813-79714dc50389 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.493227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.494296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.494582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.494938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.495603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.497600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.526584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1221/2440] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:25.531737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.543213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6e906-99b6-43ec-94ac-13f1dd8aeed3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.621856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.623229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.623618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.623964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.624703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.626968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.635636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:25.663651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1220/2441] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:25.668285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.680476 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb54c3d-ecd6-4018-8d98-aee5f08348f6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.685930 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.686689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.687212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.687498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.688613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.689803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.701751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1222/2442] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:25.707131 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.721141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d5328-8035-4bed-b700-02ca38e92d88 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.726701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.727264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.727613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.727870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.728628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.730813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aebc48-064d-4b85-b673-2d97c3d7ec50 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.742348 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1221/2443] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:25.746581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.756557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465aaf-ad5f-4efb-aaaf-e3415c3f938a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:25.762062 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:25.762760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.763045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.763325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] PATH_INFO: `/v3/projects/07324840b7904c2c8c66a14f0c24b541/users/ad5a304405e6443dbfca71581d719189/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.779684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07324840b7904c2c8c66a14f0c24b541, user_id=ad5a304405e6443dbfca71581d719189, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:25.782371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:25.795509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1223/2444] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:25 2022] PUT /identity/v3/projects/07324840b7904c2c8c66a14f0c24b541/users/ad5a304405e6443dbfca71581d719189/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:25.805638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:25.805992 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:25.806271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:25.835368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] MFA Rules not processed for user `ad5a304405e6443dbfca71581d719189`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:25.836591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:25.904419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1222/2445] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:25.916292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:25.918297 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:26.029518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd19a-2663-4769-a294-fe4addc8d2e7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:26.036122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:26.036753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:26.037037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:26.037316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:26.046920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:26.048611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59c07f-9ca9-4902-bc29-164940349688 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:26.067131 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1224/2446] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:26.326242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:26.394680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a7e8b-b467-4f76-9405-787c87e337e7 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:26.399454 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:26.400876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:26.401160 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:26.401436 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] PATH_INFO: `/v3/users/ad5a304405e6443dbfca71581d719189` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:26.402634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5a304405e6443dbfca71581d719189)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:26.404666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:26.459149 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5a304405e6443dbfca71581d719189'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:26.460485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5a304405e6443dbfca71581d719189'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:26.461248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5a304405e6443dbfca71581d719189'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:26.471254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5a304405e6443dbfca71581d719189'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:26.486306 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:26.488172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5a304405e6443dbfca71581d719189'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:26.508067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1223/2447] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:26 2022] DELETE /identity/v3/users/ad5a304405e6443dbfca71581d719189 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:26.514199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:26.514517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:26.514811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:26.535679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:26.537044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:26.612409 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1225/2448] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:26.622075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:26.624099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:26.706997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705b15-a99d-4274-8ad8-91b344eec73d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:26.711288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:26.711981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:26.713156 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:26.713577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:26.725439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:26.727487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe34fa5-ce61-455a-8a31-8a5d232be0b4 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:26.742701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1224/2449] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:27.100054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.101558 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:27.114654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9e42e-c437-45d2-81e1-4b745b4bc0ec None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.120676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.121313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.121596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.121872 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] PATH_INFO: `/v3/projects/07324840b7904c2c8c66a14f0c24b541` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.123043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07324840b7904c2c8c66a14f0c24b541)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.125154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.127423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:27.184973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07324840b7904c2c8c66a14f0c24b541', 'request_id': 'req-b9d40a57-9734-437c-bc07-bd314681b1f9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:27.190732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07324840b7904c2c8c66a14f0c24b541', 'request_id': 'req-b9d40a57-9734-437c-bc07-bd314681b1f9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:27.202602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07324840b7904c2c8c66a14f0c24b541', 'request_id': 'req-b9d40a57-9734-437c-bc07-bd314681b1f9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:27.218875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] The token cache is being invalidate because project 07324840b7904c2c8c66a14f0c24b541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:27.219312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:27.220043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b9d40a57-9734-437c-bc07-bd314681b1f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:27.221633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1226/2450] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:27 2022] DELETE /identity/v3/projects/07324840b7904c2c8c66a14f0c24b541 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:27.226626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.226958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.227242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.248237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:27.249481 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:27.326997 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1225/2451] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:27.335194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.335488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.335773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.360529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:27.361776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:27.371880 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1227/2452] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:27.376016 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.386996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e321462-e605-4667-b143-722c24d8cb00 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.392033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.392692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.392974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.393250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.393952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:43:27.394219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.395887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.409731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1226/2453] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:43:27.414133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.424260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17bcac-5bae-4376-906f-95b33f20246d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.429792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.430911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.431089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.432048 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.433185 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.434775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.467682 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1228/2454] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:27.471728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.483792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01d74-2f39-4bb8-a28f-359404166a6b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.554764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.555386 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.556651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.556902 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.557491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.559740 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.588170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1227/2455] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:27.592382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.605174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8066987-415c-4ba4-b089-af65163ebbb5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.610160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.610802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.611088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.611366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.612019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.613787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.629984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1229/2456] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:27.631302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.644795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84562bef-621c-4f91-8577-a60dba5a8bc3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.649887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.650554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.651014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.651299 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.652252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.653681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217ea88-0b3f-423c-928e-15a51856fa67 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.655755 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:27.668361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1228/2457] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:27.672216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.683667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98354b-eb40-4ead-a480-71efb1370fd5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.688854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.689548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.689836 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.690117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] PATH_INFO: `/v3/projects/fb842afaf9c3419982998eec31649132/users/82991e19ebdd4eafb92223aabef9639f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.710136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb842afaf9c3419982998eec31649132, user_id=82991e19ebdd4eafb92223aabef9639f, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.713597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:27.736871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1230/2458] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:27 2022] PUT /identity/v3/projects/fb842afaf9c3419982998eec31649132/users/82991e19ebdd4eafb92223aabef9639f/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:27.741193 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.741491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.741770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.771504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] MFA Rules not processed for user `82991e19ebdd4eafb92223aabef9639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:27.772746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:27.861726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1229/2459] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:43:27.866685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:27.964596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294dad6-4de8-49e8-9123-4ad444194b14 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:27.972523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:27.973256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:27.973666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:27.974332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:27.975263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:27.977367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.045049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1231/2460] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:27 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:28.045704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.063758 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c0a71-04ac-477d-ab92-c3d49000c005 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.193851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.194690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.194855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.195042 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.195727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.198606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.223566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1230/2461] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:28 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:43:28.227486 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.244658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffc95-0e1e-4734-a362-2622353daa01 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.249933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.250546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.250858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.251140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.251784 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.253385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.260677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:28.270462 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1232/2462] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:28.277293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.289665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cd366-55f8-4367-921f-052a9cee25af admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.297065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.298083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.298353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.298633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] PATH_INFO: `/v3/projects/049af07aabfd443bba4c562dc98be9f9/users/79ea3a03e95944ae9ea5f43b7930fcad/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.325359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=049af07aabfd443bba4c562dc98be9f9, user_id=79ea3a03e95944ae9ea5f43b7930fcad, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.329130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.348199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1231/2463] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:28 2022] PUT /identity/v3/projects/049af07aabfd443bba4c562dc98be9f9/users/79ea3a03e95944ae9ea5f43b7930fcad/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:28.351734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.366773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c61b5-83f1-4f0d-8efc-b07ffd7803b3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.439060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.440130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.440130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.440332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.441103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.443170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafb4b2-db5d-43a9-91c2-b8d5743c5151 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.456016 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1233/2464] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:28.460123 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.471535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71984b2-d8a2-4d66-ad07-06a89bdb21c3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.477150 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.477489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.478008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.478407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] PATH_INFO: `/v3/projects/049af07aabfd443bba4c562dc98be9f9/users/79ea3a03e95944ae9ea5f43b7930fcad/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.481362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=049af07aabfd443bba4c562dc98be9f9, user_id=79ea3a03e95944ae9ea5f43b7930fcad, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.484070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.496264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1232/2465] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:28 2022] PUT /identity/v3/projects/049af07aabfd443bba4c562dc98be9f9/users/79ea3a03e95944ae9ea5f43b7930fcad/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:43:28.507834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.508161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.508441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.544696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] MFA Rules not processed for user `79ea3a03e95944ae9ea5f43b7930fcad`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:28.545989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:28.636873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1234/2466] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:43:28.648415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.650824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:28.715710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:28.751676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acdf1c-581e-46d0-86fa-48d0a2aa4e60 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.756870 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.757511 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.757839 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.758114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:28.770570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:28.772228 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf790f-010c-4e0a-ba97-97202063992e service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:28.786672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1233/2467] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:28.969577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:28.987541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046867b-deac-4fd6-9049-c598aacbfbce None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:28.996684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:28.997670 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:28.998288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:28.998801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:29.112871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:29.114828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:29.131561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1235/2468] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:29.748671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:29.749202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:29.749202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:29.781206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] MFA Rules not processed for user `90be4ab702e14f4387b0883dead72d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:29.782732 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:29.861830 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1234/2469] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:29.866320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:29.867994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:29.892571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:29.941451 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b627-1703-4dc6-9bee-dae633f91775 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:29.946980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:29.947630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:29.947918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:29.948194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:29.948842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:29.950832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:29.981857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1236/2470] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:29 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:43:29.986043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:29.997254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e0b6c-33f2-4a24-a3cd-b777be98a06e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:30.060685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:30.061316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.061599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.061876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.062516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:30.064499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:30.087824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1235/2471] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:29 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:43:30.091927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:30.103072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bbbc-f2f0-4ac9-9785-5f41f931afe2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:30.108117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:30.108726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.109010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.109288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.109941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:30.111600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:30.123753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1237/2472] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:30.128129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:30.139633 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612138c8-0c6e-41d0-87d2-136f7f1c47ba admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:30.144712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:30.145374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.145657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.145931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] PATH_INFO: `/v3/projects/18adf863101244fe954a347593663444/users/41b4bacdcd76476d8ef084eeb58f2569/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.163003 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18adf863101244fe954a347593663444, user_id=41b4bacdcd76476d8ef084eeb58f2569, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:30.166292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:30.180141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1236/2473] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:30 2022] PUT /identity/v3/projects/18adf863101244fe954a347593663444/users/41b4bacdcd76476d8ef084eeb58f2569/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:30.184782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:30.195411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c7fb-4db1-4a82-9ea9-aa0c5daa9f15 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:30.255356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:30.255965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.256310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.256869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.257517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:30.260557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604975f-6bfa-42a6-9316-b720aea0b633 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:30.276029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1238/2474] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:30.277472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:30.288332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dd89a-ea8b-4053-a752-6c934e2446f5 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:30.293312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:30.293919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.294326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.294612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] PATH_INFO: `/v3/projects/18adf863101244fe954a347593663444/users/41b4bacdcd76476d8ef084eeb58f2569/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.296614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18adf863101244fe954a347593663444, user_id=41b4bacdcd76476d8ef084eeb58f2569, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:30.299562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:30.312157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1237/2475] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:30 2022] PUT /identity/v3/projects/18adf863101244fe954a347593663444/users/41b4bacdcd76476d8ef084eeb58f2569/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:30.379630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11207-b257-470a-ac3b-179095aeccee None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:30.379947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11207-b257-470a-ac3b-179095aeccee None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:30.380216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11207-b257-470a-ac3b-179095aeccee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:30.402980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8de11207-b257-470a-ac3b-179095aeccee None None] MFA Rules not processed for user `81be4c83d3b04624ae11bcbf37a44309`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:30.404205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8de11207-b257-470a-ac3b-179095aeccee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:30.466569 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1239/2476] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:31.933478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:31.968375 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:32.017053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be229f4-9af0-48ea-a369-5c7ff07a0e72 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:32.022294 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:32.023235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:32.023587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:32.023927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] PATH_INFO: `/v3/users/82991e19ebdd4eafb92223aabef9639f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:32.025937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82991e19ebdd4eafb92223aabef9639f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:32.028600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:32.113940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82991e19ebdd4eafb92223aabef9639f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:32.114832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82991e19ebdd4eafb92223aabef9639f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:32.115578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82991e19ebdd4eafb92223aabef9639f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:32.127324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82991e19ebdd4eafb92223aabef9639f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:32.147345 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82991e19ebdd4eafb92223aabef9639f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:32.167387 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1238/2477] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:31 2022] DELETE /identity/v3/users/82991e19ebdd4eafb92223aabef9639f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:32.172263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:32.173101 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:32.174022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:32.199937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:32.201818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:32.246326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:32.290684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1240/2478] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:32.307644 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:32.309443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:32.396737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb9e7f-afa3-4165-a931-5be445d07de8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:32.401540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:32.402686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:32.402686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:32.402686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:32.417620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:32.419463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:32.431329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1239/2479] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:32.876179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:32.877812 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:32.899949 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d0e14-8cd0-46a5-a2d8-bce8aed65dd2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:32.909050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:32.909415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:32.909695 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:32.909985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] PATH_INFO: `/v3/projects/fb842afaf9c3419982998eec31649132` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:32.911715 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb842afaf9c3419982998eec31649132)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:32.927427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:33.001951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb842afaf9c3419982998eec31649132', 'request_id': 'req-d00856af-047a-4611-99dd-a650de2e78d5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.008014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb842afaf9c3419982998eec31649132', 'request_id': 'req-d00856af-047a-4611-99dd-a650de2e78d5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.015238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb842afaf9c3419982998eec31649132', 'request_id': 'req-d00856af-047a-4611-99dd-a650de2e78d5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.050823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] The token cache is being invalidate because project fb842afaf9c3419982998eec31649132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:33.051136 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.051796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.053711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1241/2480] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:32 2022] DELETE /identity/v3/projects/fb842afaf9c3419982998eec31649132 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:33.057982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:33.070441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4336a-a938-492e-a321-88192a18dea8 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:33.138427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:33.175805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:33.176547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:33.177197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:33.177579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] PATH_INFO: `/v3/users/79ea3a03e95944ae9ea5f43b7930fcad` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:33.179497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ea3a03e95944ae9ea5f43b7930fcad)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:33.182287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:33.247736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ea3a03e95944ae9ea5f43b7930fcad'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.250073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ea3a03e95944ae9ea5f43b7930fcad'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.251339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ea3a03e95944ae9ea5f43b7930fcad'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.263402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ea3a03e95944ae9ea5f43b7930fcad'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.285524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-528b825f-921a-4e38-9624-23d966ef04b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ea3a03e95944ae9ea5f43b7930fcad'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:33.302428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1240/2481] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:43:33 2022] DELETE /identity/v3/users/79ea3a03e95944ae9ea5f43b7930fcad => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:34.505122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:34.506690 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:34.531381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00856af-047a-4611-99dd-a650de2e78d5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:34.662305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:34.663135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:34.663421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:34.663840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] PATH_INFO: `/v3/projects/049af07aabfd443bba4c562dc98be9f9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:34.666076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=049af07aabfd443bba4c562dc98be9f9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:34.668974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:34.747274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049af07aabfd443bba4c562dc98be9f9', 'request_id': 'req-7eab4478-d8ac-4986-b21f-048b241e4ff2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:34.762004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049af07aabfd443bba4c562dc98be9f9', 'request_id': 'req-7eab4478-d8ac-4986-b21f-048b241e4ff2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:34.765372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:34.787791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049af07aabfd443bba4c562dc98be9f9', 'request_id': 'req-7eab4478-d8ac-4986-b21f-048b241e4ff2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:34.817890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] The token cache is being invalidate because project 049af07aabfd443bba4c562dc98be9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:43:34.818188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:34.818926 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-7eab4478-d8ac-4986-b21f-048b241e4ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:43:34.822726 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1242/2482] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:43:34 2022] DELETE /identity/v3/projects/049af07aabfd443bba4c562dc98be9f9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:34.826926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:34.827433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:34.827560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:34.860896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:34.863672 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:35.013690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1241/2483] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:35.023018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.023550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.024924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.057214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:35.058814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:35.097020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1243/2484] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:35.098425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.114599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d81810-cee3-499f-b3b9-8b9dd9259737 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.128062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.128815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.129207 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.129589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.130364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:43:35.134127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.136318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.166104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1242/2485] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:43:35.171737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.192579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21425-9def-45e1-9f95-c45492e42f7e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.211610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.212429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.212825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.213218 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.214055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.216277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.280706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1244/2486] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:43:35 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:35.285196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.300842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4cff3-bfa1-4be5-85a1-66b37a1fb377 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.421569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:35.438539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.439606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.439968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.440318 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.441288 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.443720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.482917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.483141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1243/2487] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:43:35 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:43:35.495213 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076507-5715-4d3d-a12f-83bf220c0115 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.501408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.502270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.502918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.503377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.504192 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.506038 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.526543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1245/2488] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:43:35.530900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.552842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e9027-ee25-4179-88fe-2c578d9403f0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.560850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.561605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.562166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.562373 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.563216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.565543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d82fe-5926-4a52-b497-210406261be1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.597023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1244/2489] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:43:35.598513 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.611829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea67f19-2f2f-45f5-a9c1-90186068d791 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.618477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.620207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.620489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.621155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] PATH_INFO: `/v3/projects/c281c7bb8760439c8cabf6981e7a1862/users/1b50ba7bd1c84dbf849820a94dfdad47/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.643159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c281c7bb8760439c8cabf6981e7a1862, user_id=1b50ba7bd1c84dbf849820a94dfdad47, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.646581 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.662410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1246/2490] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:43:35 2022] PUT /identity/v3/projects/c281c7bb8760439c8cabf6981e7a1862/users/1b50ba7bd1c84dbf849820a94dfdad47/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:43:35.675724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e4ff-87ff-415d-9e8e-4221ea2a8645 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.676020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e4ff-87ff-415d-9e8e-4221ea2a8645 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.676314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695e4ff-87ff-415d-9e8e-4221ea2a8645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.714342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-c695e4ff-87ff-415d-9e8e-4221ea2a8645 None None] MFA Rules not processed for user `1b50ba7bd1c84dbf849820a94dfdad47`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:35.715721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c695e4ff-87ff-415d-9e8e-4221ea2a8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:35.841398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1245/2491] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:35.855466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:35.857389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:35.943006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c97117-812a-4337-9bf8-ca58f35d949f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:35.950209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:35.950796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:35.951086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:35.951369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:35.964263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:35.965961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:35.977931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1247/2492] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:37.596525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6a11-13e6-4c01-8e6f-cb4d67c2ba32 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:37.597026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6a11-13e6-4c01-8e6f-cb4d67c2ba32 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:37.597111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a6a11-13e6-4c01-8e6f-cb4d67c2ba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:37.628235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9a4a6a11-13e6-4c01-8e6f-cb4d67c2ba32 None None] MFA Rules not processed for user `41b4bacdcd76476d8ef084eeb58f2569`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:37.629468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9a4a6a11-13e6-4c01-8e6f-cb4d67c2ba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:37.716320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1246/2493] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:37.730999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:37.732758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:37.816969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68430b62-46bd-4191-a003-7db1b6ad5ecd service glance] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:37.825761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:37.826541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:37.826983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:37.827506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:37.829468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:37.872754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:37.913898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:37.916509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:37.928758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1248/2494] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:39.291792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:39.292702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:39.293200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:39.323435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] MFA Rules not processed for user `90be4ab702e14f4387b0883dead72d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:43:39.325011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:39.334015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1247/2495] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:43:47.923875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=16 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:49.313513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:50.430833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:50.432303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:50.498123 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f897d5-cfd9-4b19-85b1-4b87b492f462 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:50.501866 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:50.502561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:50.502882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:50.503160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:50.505748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:50.587559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:50.588937 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:50.598447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1249/2496] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:00.594035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=50 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:06.293606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:06.306663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32998-7603-4582-9d3b-0ea2c62ddc64 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:06.311192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:06.311827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:06.312105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:06.312376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:06.314567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:06.386588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:06.388062 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:06.399890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1248/2497] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:10.205696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.207255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.221788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45bda2-eeb3-4f09-b1b7-8b641cf534be service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.227734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.228487 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.228899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.229266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] PATH_INFO: `/v3/users/1541e1614d664891b1952379b003ed71` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.230525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1541e1614d664891b1952379b003ed71)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.233570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.293739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1541e1614d664891b1952379b003ed71'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.294678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1541e1614d664891b1952379b003ed71'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.302461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1541e1614d664891b1952379b003ed71'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.304076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1541e1614d664891b1952379b003ed71'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.322985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c6b8bff-b966-4a7d-9c03-26c639170b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1541e1614d664891b1952379b003ed71'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.345911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1250/2498] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:10 2022] DELETE /identity/v3/users/1541e1614d664891b1952379b003ed71 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:10.523258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.535798 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e50a50-7ce4-45d3-a3a7-f78b082eb52f service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.600001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.600719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.601001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.601275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] PATH_INFO: `/v3/projects/0b8dd47a9d30492eb7163da543d874cb` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.602721 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8dd47a9d30492eb7163da543d874cb)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.604637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.649953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8dd47a9d30492eb7163da543d874cb', 'request_id': 'req-5eb72211-d91f-4963-815b-c78d9f0220e1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.655471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8dd47a9d30492eb7163da543d874cb', 'request_id': 'req-5eb72211-d91f-4963-815b-c78d9f0220e1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.661437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8dd47a9d30492eb7163da543d874cb', 'request_id': 'req-5eb72211-d91f-4963-815b-c78d9f0220e1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.674226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] The token cache is being invalidate because project 0b8dd47a9d30492eb7163da543d874cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:10.674516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.675208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-5eb72211-d91f-4963-815b-c78d9f0220e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.676696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1249/2499] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:10 2022] DELETE /identity/v3/projects/0b8dd47a9d30492eb7163da543d874cb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:10.682389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.682718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.683002 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.702413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:10.703668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.771015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1251/2500] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:10.780730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.781455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.781907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.804399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:10.805771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.814096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1250/2501] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:10.818458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.828981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b537e-e1e5-45cf-91a8-d746f5330680 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.833868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.834550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.835576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.835859 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.836521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:10.836827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.838434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.851179 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1252/2502] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:10.856675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.866955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708fe28-71b5-4825-95a0-f38e1ad23ff2 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.872054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.872667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.872976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.873268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.873885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.875890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.893515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:10.909059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1251/2503] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:10 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:10.913024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.923714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862ff57-630d-4516-9fb9-0128c867c9d0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.986454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.987269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.987552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.987828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.988461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.990386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.998663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:11.014221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1253/2504] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:10 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:11.018384 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.030435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617899-3f01-4b67-84bc-efc5576bf505 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.035250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.035876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.036153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.036421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.037060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.038660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.048788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1252/2505] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:11.052785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.063373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab3e15-6d9d-4cf0-9b53-96a64c5dbc5a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.067964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.068553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.068839 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.069801 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.070353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.071987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8035-a21d-4de0-af39-a2086b557b74 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.083672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1254/2506] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:11.088151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.103254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39f8f-b787-48f3-bc64-1098582175d7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.108022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.108683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.108960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.109231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] PATH_INFO: `/v3/projects/8cc56d9ec152477db041647ee1ff1b2a/users/d667576f13d2435e99df66fac5dfe5c6/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.127027 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc56d9ec152477db041647ee1ff1b2a, user_id=d667576f13d2435e99df66fac5dfe5c6, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.129955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.145289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1253/2507] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:11 2022] PUT /identity/v3/projects/8cc56d9ec152477db041647ee1ff1b2a/users/d667576f13d2435e99df66fac5dfe5c6/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:11.156094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.156386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.156667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.184542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] MFA Rules not processed for user `d667576f13d2435e99df66fac5dfe5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:11.185760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.252321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1255/2508] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:11.263730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.265441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.344514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4bf65-28d5-4117-8a0b-b25085a7b521 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.349979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.351311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.351715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.351936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.360247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.361822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c3a41-8515-4bc8-9f33-d891ecd72ddd service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.371422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1254/2509] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:14.110119 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:14.174610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309cfe7-2e1b-46ef-be6e-63f27537ccfd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:14.179197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:14.179810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.180089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.180362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] PATH_INFO: `/v3/users/d667576f13d2435e99df66fac5dfe5c6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.181468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d667576f13d2435e99df66fac5dfe5c6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:14.183439 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:14.229404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd667576f13d2435e99df66fac5dfe5c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.230063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd667576f13d2435e99df66fac5dfe5c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.242163 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd667576f13d2435e99df66fac5dfe5c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.242795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd667576f13d2435e99df66fac5dfe5c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.250008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd667576f13d2435e99df66fac5dfe5c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.255190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:14.269057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1256/2510] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:14 2022] DELETE /identity/v3/users/d667576f13d2435e99df66fac5dfe5c6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:14.274013 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.274523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.274631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.293291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:14.294470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:14.357596 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1255/2511] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:14.367740 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:14.369199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:14.430468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a496-7c7c-44cc-b19e-a22657ef60d5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:14.434180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:14.434822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.435108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.435382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.445825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:14.447320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45090712-5729-4e8d-8fdb-f69031836e7f service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:14.457023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1257/2512] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:14.730927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:14.735516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:14.750613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd692f-1704-4fa1-819c-78c6fb3b8b33 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:14.755326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:14.755979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.756852 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.757198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] PATH_INFO: `/v3/projects/8cc56d9ec152477db041647ee1ff1b2a` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.758355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc56d9ec152477db041647ee1ff1b2a)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:14.760316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:14.801756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:14.809192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc56d9ec152477db041647ee1ff1b2a', 'request_id': 'req-c2575c3a-d97c-43e0-bc97-9e6901f88f82'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.814982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc56d9ec152477db041647ee1ff1b2a', 'request_id': 'req-c2575c3a-d97c-43e0-bc97-9e6901f88f82'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.821789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc56d9ec152477db041647ee1ff1b2a', 'request_id': 'req-c2575c3a-d97c-43e0-bc97-9e6901f88f82'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.833890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] The token cache is being invalidate because project 8cc56d9ec152477db041647ee1ff1b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:14.834197 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.835061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c2575c3a-d97c-43e0-bc97-9e6901f88f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:14.836601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1256/2513] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:14 2022] DELETE /identity/v3/projects/8cc56d9ec152477db041647ee1ff1b2a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:14.841440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.841731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.842005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.861186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:14.862381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:14.922958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1258/2514] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:14.931430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.931719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.931998 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.952597 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:14.953840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:14.961982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1257/2515] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:14.966274 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:14.976228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cb4b5-e2fb-499f-9130-9159115dacb1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:14.981343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:14.981935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:14.982214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:14.982485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:14.983158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:14.983458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:14.985059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:14.998242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1259/2516] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:15.002403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.012866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac8d3f-5e52-4f0b-ac24-cdc2fd8fe920 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.018010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.018863 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.019008 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.019209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.019881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.021815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.053460 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1258/2517] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:15.056853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.067376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c02f-9737-46a4-ac11-2ae9cb0c5f44 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.126531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.127176 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.127453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.127724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.128351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.130268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.159346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1260/2518] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:15.166823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.174862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372a8c9-446a-4b6d-940a-42598a8c9496 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.180435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.181040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.181321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.181591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.182233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.183850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.198714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1259/2519] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:15.198910 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.210220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca802b-c17d-4fce-bb0e-5431a704de72 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.215444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.216089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.216477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.216641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.217313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.219461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5d7b7-6dcb-4c56-b10c-519895ad8ec4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.220567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:15.240589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1261/2520] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:15.241693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.252684 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f63d-916c-4469-b9bc-c55749cf560a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.260328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.261001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.261285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.261563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] PATH_INFO: `/v3/projects/8996036c6ee64e1c958fa2003bef9646/users/e82539beacb8427fa335e00f2a778383/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.278810 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8996036c6ee64e1c958fa2003bef9646, user_id=e82539beacb8427fa335e00f2a778383, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.281523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.296891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1260/2521] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:15 2022] PUT /identity/v3/projects/8996036c6ee64e1c958fa2003bef9646/users/e82539beacb8427fa335e00f2a778383/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:15.308677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.308897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.309175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.338314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] MFA Rules not processed for user `e82539beacb8427fa335e00f2a778383`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:15.339565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:15.405194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1262/2522] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:15.409751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.476409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712238fa-1725-456f-ac6c-f71954b04b0d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.481457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.482075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.482352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.482626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.483278 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.485168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.512697 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1261/2523] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:15.516775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.528420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c001d-2bf9-41d7-aea3-50b52a680f78 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.585942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.586468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.586792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.587069 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.587725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.589653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.611935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1263/2524] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:15.617097 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.627688 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe3596-df15-447f-ae7a-3be4ba2b5725 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.633130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.633731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.634017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.634295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.634989 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.636577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.647245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1262/2525] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:15.651452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.661283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e19-1f01-4540-ab80-4d53abc33b4a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.666178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.666854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.667140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.667420 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] PATH_INFO: `/v3/projects/c7ffcf8f6197430597d357b4a8cae12b/users/66ba5e7d95c04184a7646450dd0b4920/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.682495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ffcf8f6197430597d357b4a8cae12b, user_id=66ba5e7d95c04184a7646450dd0b4920, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.685190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.698814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1264/2526] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:15 2022] PUT /identity/v3/projects/c7ffcf8f6197430597d357b4a8cae12b/users/66ba5e7d95c04184a7646450dd0b4920/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:15.702731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.712837 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1310d1-8402-44a5-ab9d-6d6a3b778844 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.752465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:15.772932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.773542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.773838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.774145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.774832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.776420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd6a0f-4a68-45bb-8bf6-ca49f7b8070c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.786060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1263/2527] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:15.790290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.800903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e8821-7f23-4b63-a953-ecdd487d04eb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.807508 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.808370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.808729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.809008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] PATH_INFO: `/v3/projects/c7ffcf8f6197430597d357b4a8cae12b/users/66ba5e7d95c04184a7646450dd0b4920/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.811603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ffcf8f6197430597d357b4a8cae12b, user_id=66ba5e7d95c04184a7646450dd0b4920, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.814504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.827170 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1265/2528] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:15 2022] PUT /identity/v3/projects/c7ffcf8f6197430597d357b4a8cae12b/users/66ba5e7d95c04184a7646450dd0b4920/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:15.837656 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.837877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.838157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.867358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] MFA Rules not processed for user `66ba5e7d95c04184a7646450dd0b4920`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:15.868639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:15.945005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1264/2529] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:15.956334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.958014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:16.001419 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:16.017335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd19a9-67a4-46a4-8971-56a4198b1a48 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:16.020953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:16.021550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.021857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.022124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.031320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:16.032745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:16.042555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1266/2530] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:16.128296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:16.138145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5172557-2576-411c-b2d7-276549ebb37c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:16.141736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:16.142329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.142608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.142913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.204571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:16.206010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20e0d9-c129-479c-91f4-07677ebaa15b service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:16.215441 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1265/2531] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:16.324982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:16.326433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:16.410177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a44a4-fa15-4575-8a77-bd547dc4b1dd service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:16.415683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:16.416311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.416595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.416867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] PATH_INFO: `/v3/users/1b50ba7bd1c84dbf849820a94dfdad47` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.417982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b50ba7bd1c84dbf849820a94dfdad47)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:16.419945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:16.473772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b50ba7bd1c84dbf849820a94dfdad47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:16.474410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b50ba7bd1c84dbf849820a94dfdad47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:16.482259 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b50ba7bd1c84dbf849820a94dfdad47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:16.483042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b50ba7bd1c84dbf849820a94dfdad47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:16.490489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b50ba7bd1c84dbf849820a94dfdad47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:16.508127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1267/2532] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:16 2022] DELETE /identity/v3/users/1b50ba7bd1c84dbf849820a94dfdad47 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:16.512707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.513185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.513303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.533548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:16.534795 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:16.585243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:16.594086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1266/2533] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:16.604860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:16.606259 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:16.663539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c92f-6703-48a1-8abd-ff180f4d816a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:16.667142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:16.667741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.668021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.668296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.677132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:16.678777 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28212c34-5a49-4e70-aa2b-eca1ef1ca65f service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:16.688167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1268/2534] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:16.964370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:16.965848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:16.979184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e1db7-6eab-414f-b5d9-0f7f72b45708 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:16.983817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:16.984467 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:16.984729 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:16.985006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] PATH_INFO: `/v3/projects/c281c7bb8760439c8cabf6981e7a1862` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:16.986142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c281c7bb8760439c8cabf6981e7a1862)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:16.988142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.036772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c281c7bb8760439c8cabf6981e7a1862', 'request_id': 'req-17a93f05-3062-4947-89a3-0a10315e6841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:17.042447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c281c7bb8760439c8cabf6981e7a1862', 'request_id': 'req-17a93f05-3062-4947-89a3-0a10315e6841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:17.048635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c281c7bb8760439c8cabf6981e7a1862', 'request_id': 'req-17a93f05-3062-4947-89a3-0a10315e6841'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:17.061785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] The token cache is being invalidate because project c281c7bb8760439c8cabf6981e7a1862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:17.062076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:17.062751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-17a93f05-3062-4947-89a3-0a10315e6841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:17.064300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1267/2535] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:16 2022] DELETE /identity/v3/projects/c281c7bb8760439c8cabf6981e7a1862 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:17.070455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.070773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.071050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.079593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:17.092797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:17.094005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:17.153961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1269/2536] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:17.161610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.161901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.162182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.182340 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:17.183593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:17.191898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1268/2537] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:17.197151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.207012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a27ab-5976-483d-9bd6-8718a6c26145 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.211928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.212527 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.212810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.213082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.213734 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:17.214041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.215703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.227433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1270/2538] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:17.231707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.241586 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b027e4b-89c5-4e92-ad16-886533cba64f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.246301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.246943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.247361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.247642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.248266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.250274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.281843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1269/2539] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:17.283120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.293196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42fe11-a5b9-4f51-927d-462149900af5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.353401 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.354128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.354423 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.354992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.355382 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.357363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.382831 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1271/2540] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:17.384036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.394984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd53b69-0294-40a8-9c9d-37a97042bf8e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.401324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.401937 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.402514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.402514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.403176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.405104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.416199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1270/2541] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:17.420355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.441292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e247-35e8-4d2d-a596-31c06b8cdcf9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.446725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.447202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.447577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.447741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.448385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.449979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09365b-116b-469a-8152-20fa5c32ea07 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.462047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1272/2542] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:17.466010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.476446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b7390-7ab3-47f6-87e6-ad4704909bee admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.481061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.482049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.482049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.482293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] PATH_INFO: `/v3/projects/0ce10007c6874c1b92aa348c682fc043/users/9d6d7f575304448a89d06e25ccd2d119/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.498241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce10007c6874c1b92aa348c682fc043, user_id=9d6d7f575304448a89d06e25ccd2d119, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.500960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.524523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1271/2543] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:17 2022] PUT /identity/v3/projects/0ce10007c6874c1b92aa348c682fc043/users/9d6d7f575304448a89d06e25ccd2d119/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:17.534490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87bab-e6d8-474c-9dc6-b1d554c30588 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.534810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87bab-e6d8-474c-9dc6-b1d554c30588 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.535086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87bab-e6d8-474c-9dc6-b1d554c30588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.569694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a3b87bab-e6d8-474c-9dc6-b1d554c30588 None None] MFA Rules not processed for user `9d6d7f575304448a89d06e25ccd2d119`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:17.571458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a3b87bab-e6d8-474c-9dc6-b1d554c30588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:17.647411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:17.654772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1273/2544] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:17.667750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:17.669585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:17.738223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a77da-7e1e-435e-84af-83e81c22bdf9 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:17.742982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:17.743632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.743980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.744324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.753786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:17.755519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:17.767675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1272/2545] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:19.127005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:19.127890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:19.128324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:19.154108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:19.154786 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:19.223843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1274/2546] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:19.279151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:19.280627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:19.307866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:19.358546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8fc2-11f0-4b78-862b-849cf0a66952 service glance] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:19.363555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:19.364165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:19.364500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:19.364775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:19.374286 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:19.375944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d075c20-33fa-4ff1-8966-27cb9749fca3 service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:19.385573 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1273/2547] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:44:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:44:26.850039 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:26.851658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:26.934421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd063ad-abe2-4775-9a33-f54388e6455a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:26.942633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:26.944265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:26.944564 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:26.944915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] PATH_INFO: `/v3/users/e82539beacb8427fa335e00f2a778383` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:26.946158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82539beacb8427fa335e00f2a778383)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:26.948472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:27.007099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82539beacb8427fa335e00f2a778383'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.007844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82539beacb8427fa335e00f2a778383'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.015227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82539beacb8427fa335e00f2a778383'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.015970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82539beacb8427fa335e00f2a778383'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.025745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82539beacb8427fa335e00f2a778383'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.048788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1275/2548] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:26 2022] DELETE /identity/v3/users/e82539beacb8427fa335e00f2a778383 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:27.053572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:27.054534 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:27.054936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:27.077081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:27.078437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:27.162772 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1274/2549] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:27.173341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:27.175012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:27.206950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:27.252913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d45f5-c3e6-4277-92d4-36ad62a66d73 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:27.257497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:27.258109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:27.258390 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:27.258691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:27.270762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:27.272882 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:27.292655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1276/2550] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:27.871449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:27.873222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:27.890898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7c84c-b05d-46b9-89f6-8d970a492f2d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:27.898492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:27.906702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:27.906702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:27.906702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] PATH_INFO: `/v3/projects/8996036c6ee64e1c958fa2003bef9646` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:27.906702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8996036c6ee64e1c958fa2003bef9646)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:27.906702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:27.980325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8996036c6ee64e1c958fa2003bef9646', 'request_id': 'req-3747bf31-ff35-473a-a86a-40d340b77944'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.987794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8996036c6ee64e1c958fa2003bef9646', 'request_id': 'req-3747bf31-ff35-473a-a86a-40d340b77944'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:27.994993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8996036c6ee64e1c958fa2003bef9646', 'request_id': 'req-3747bf31-ff35-473a-a86a-40d340b77944'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.016010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] The token cache is being invalidate because project 8996036c6ee64e1c958fa2003bef9646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:28.016298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.016923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.018690 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1275/2551] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:27 2022] DELETE /identity/v3/projects/8996036c6ee64e1c958fa2003bef9646 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:28.024141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:28.038042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c28e-859b-4ca3-b2e1-bd8623692d69 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:28.113084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:28.113876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:28.114256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:28.114625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] PATH_INFO: `/v3/users/66ba5e7d95c04184a7646450dd0b4920` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:28.116288 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ba5e7d95c04184a7646450dd0b4920)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:28.118596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:28.172967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ba5e7d95c04184a7646450dd0b4920'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.173647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ba5e7d95c04184a7646450dd0b4920'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.181728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ba5e7d95c04184a7646450dd0b4920'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.182480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ba5e7d95c04184a7646450dd0b4920'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.191828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-74cd8471-f6a4-469d-81e7-7dbfb7e95971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ba5e7d95c04184a7646450dd0b4920'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.215036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1277/2552] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:28 2022] DELETE /identity/v3/users/66ba5e7d95c04184a7646450dd0b4920 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:28.581587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:28.584371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:28.618493 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747bf31-ff35-473a-a86a-40d340b77944 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:28.673085 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:28.721503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:28.721503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:28.721503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:28.721503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] PATH_INFO: `/v3/projects/c7ffcf8f6197430597d357b4a8cae12b` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:28.724599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ffcf8f6197430597d357b4a8cae12b)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:28.727072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:28.792118 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ffcf8f6197430597d357b4a8cae12b', 'request_id': 'req-943b0c63-2bc7-4249-bd7e-8ad181cece04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.798958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ffcf8f6197430597d357b4a8cae12b', 'request_id': 'req-943b0c63-2bc7-4249-bd7e-8ad181cece04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.841487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ffcf8f6197430597d357b4a8cae12b', 'request_id': 'req-943b0c63-2bc7-4249-bd7e-8ad181cece04'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.857259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] The token cache is being invalidate because project c7ffcf8f6197430597d357b4a8cae12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:28.857563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.859018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-943b0c63-2bc7-4249-bd7e-8ad181cece04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:28.861425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1276/2553] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:28 2022] DELETE /identity/v3/projects/c7ffcf8f6197430597d357b4a8cae12b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:28.866349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:28.867457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:28.867457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:28.893595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:28.894864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:28.973444 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:29.014630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1278/2554] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:29.024455 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.024754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.025039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.048402 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:29.049894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:29.070678 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1277/2555] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:29.078675 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.085088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc573eb6-e31d-4372-9d9e-8be402bdd548 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.090934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.091772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.092208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.092835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.094146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:29.094612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.096430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.125817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1279/2556] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:29.126070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.134257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4c893-ad9e-4bf8-bcd7-12d2c0061c24 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.152925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.153890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.154022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.154411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.155432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.157600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.202807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.204400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1278/2557] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:29.213616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b8a7-f28b-4d13-a6eb-264a68277191 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.288397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.289718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.290010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.290660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.291332 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.293710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.328765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1280/2558] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:29.331133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.336925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.338915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:29.343459 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77285a-e156-4d5b-b98e-7b09280c2ac4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.348779 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.350201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.350598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.350912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.351570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.353419 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.364179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1279/2559] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:29.368143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.377803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dab3df-7fba-43d2-a2de-3343ac4b602c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.383077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.383676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.383967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.384243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.384881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.386510 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.397566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1280/2560] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:29.401804 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.404943 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7b6d7-2ab9-443e-a814-c998d4e401f0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.408588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.409200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.409481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.409755 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.412774 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:29.416430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b013d-0413-4b8a-b27e-168a5ee8c05f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.421854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.422558 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.422896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.423171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] PATH_INFO: `/v3/projects/1d4eff9de3ff4290871eea8fbee1b0b7/users/156336809ca94e0f85a008438d3300e4/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.438532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4eff9de3ff4290871eea8fbee1b0b7, user_id=156336809ca94e0f85a008438d3300e4, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.441079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31065d7a-e537-4178-8ab0-4dc8edeea216 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.456559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1281/2561] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:29 2022] PUT /identity/v3/projects/1d4eff9de3ff4290871eea8fbee1b0b7/users/156336809ca94e0f85a008438d3300e4/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:29.467339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.467626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.467900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.479181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.480624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.490437 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1281/2562] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:29.496578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] MFA Rules not processed for user `156336809ca94e0f85a008438d3300e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:29.497832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:29.540557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:29.571041 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1282/2563] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:29.575703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.587663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba92b-ac16-4b68-80c1-ea80a5639585 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.595383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:29.645324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.646772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.647260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.647548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.648174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.650093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.678602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1282/2564] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:29.682777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.694448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939bbcc-20cf-4f65-9736-2d167c41e168 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.765705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.766324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.766606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.766923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.767576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.769502 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.798865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1283/2565] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:29 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:29.800310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.812186 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13519b-4945-45ed-b29c-6efe8f00ac0e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.818066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.818897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.819265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.819599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.820342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.822004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.833020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1283/2566] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:29.838304 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.848515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a81be6-1471-44ab-a40f-d85b20f6faa7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.854276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.854987 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.855269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.855547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] PATH_INFO: `/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4/users/2f87f43f34734d5b9e76744a01084cee/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.871004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1658ed9b107a46e8bd8fd6d3c9978ad4, user_id=2f87f43f34734d5b9e76744a01084cee, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.873790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.891206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1284/2567] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:29 2022] PUT /identity/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4/users/2f87f43f34734d5b9e76744a01084cee/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:29.896571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:29.907862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9c68-06d2-4f78-81bb-2d7ee66ff050 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:29.971209 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:29.971819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:29.972099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:29.972374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:29.973014 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:29.974843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91073d-151f-4f3a-8651-aa267b3e3a22 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:29.991468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1284/2568] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:29.992694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:30.002857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca8b8-be20-4b93-8487-1f97441ba116 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:30.008484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:30.009144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.009423 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.009693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] PATH_INFO: `/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4/users/2f87f43f34734d5b9e76744a01084cee/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.012119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1658ed9b107a46e8bd8fd6d3c9978ad4, user_id=2f87f43f34734d5b9e76744a01084cee, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:30.015710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:30.029432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1285/2569] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:29 2022] PUT /identity/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4/users/2f87f43f34734d5b9e76744a01084cee/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:30.043532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.043833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.044118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.081970 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] MFA Rules not processed for user `2f87f43f34734d5b9e76744a01084cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:30.083356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:30.168481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1285/2570] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:30.180708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:30.183320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:30.257966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5044b-33ff-4397-8c46-98ed14099c20 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:30.262157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:30.262815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.263100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.263374 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.277220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:30.278817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3b9d4-c817-475d-87fc-9b703d047c21 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:30.289862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1286/2571] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:30.434534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:30.444194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:30.500976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1546c42-804e-4ac9-8802-c438a691cb66 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:30.506430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:30.507112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.507396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.507674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] PATH_INFO: `/v3/users/156336809ca94e0f85a008438d3300e4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.508826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] RBAC: Authorizing `identity:delete_user(user_id=156336809ca94e0f85a008438d3300e4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:30.510862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:30.562520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156336809ca94e0f85a008438d3300e4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:30.563225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156336809ca94e0f85a008438d3300e4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:30.570791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156336809ca94e0f85a008438d3300e4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:30.571436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156336809ca94e0f85a008438d3300e4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:30.579557 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156336809ca94e0f85a008438d3300e4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:30.603939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1286/2572] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:30 2022] DELETE /identity/v3/users/156336809ca94e0f85a008438d3300e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:30.606342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.606744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.607185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.625877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:30.627132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:30.659844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:30.691929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1287/2573] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:30.702458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:30.703919 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:30.766049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d929a-09a0-445a-81cb-63eebe28267a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:30.769811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:30.770415 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:30.771291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:30.771742 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:30.781144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:30.782626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:30.793237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1287/2574] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:31.089707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:31.091296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:31.109370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629812a-da26-46b7-a962-450e64089dd6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:31.116685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:31.117921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:31.118777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:31.119274 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] PATH_INFO: `/v3/projects/1d4eff9de3ff4290871eea8fbee1b0b7` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:31.120875 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4eff9de3ff4290871eea8fbee1b0b7)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:31.123550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:31.179141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4eff9de3ff4290871eea8fbee1b0b7', 'request_id': 'req-28da66d1-9b89-4bf4-a8d7-13349920963d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.184976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4eff9de3ff4290871eea8fbee1b0b7', 'request_id': 'req-28da66d1-9b89-4bf4-a8d7-13349920963d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.192357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4eff9de3ff4290871eea8fbee1b0b7', 'request_id': 'req-28da66d1-9b89-4bf4-a8d7-13349920963d'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.206217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] The token cache is being invalidate because project 1d4eff9de3ff4290871eea8fbee1b0b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:31.206503 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.207144 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.209568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1288/2575] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:31 2022] DELETE /identity/v3/projects/1d4eff9de3ff4290871eea8fbee1b0b7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:31.213554 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:31.223797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c77d8-8d6e-417c-9fd9-39b27e34173e service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:31.251911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:31.290007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:31.290665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:31.290961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:31.291854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] PATH_INFO: `/v3/users/2f87f43f34734d5b9e76744a01084cee` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:31.293022 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f87f43f34734d5b9e76744a01084cee)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:31.294995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:31.354246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f87f43f34734d5b9e76744a01084cee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.355658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f87f43f34734d5b9e76744a01084cee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.363107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f87f43f34734d5b9e76744a01084cee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.364477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f87f43f34734d5b9e76744a01084cee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.375055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e289b307-8b84-4980-b4f9-8ebfde4422be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f87f43f34734d5b9e76744a01084cee'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.400188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1288/2576] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:31 2022] DELETE /identity/v3/users/2f87f43f34734d5b9e76744a01084cee => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:31.831290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:31.833439 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:31.846603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da66d1-9b89-4bf4-a8d7-13349920963d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:31.905886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:31.906610 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:31.907004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:31.907279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] PATH_INFO: `/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:31.908418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1658ed9b107a46e8bd8fd6d3c9978ad4)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:31.910361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:31.960628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1658ed9b107a46e8bd8fd6d3c9978ad4', 'request_id': 'req-c33b87f8-1004-402c-ad4b-83d8a353d834'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.966542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1658ed9b107a46e8bd8fd6d3c9978ad4', 'request_id': 'req-c33b87f8-1004-402c-ad4b-83d8a353d834'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.968494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:31.973137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1658ed9b107a46e8bd8fd6d3c9978ad4', 'request_id': 'req-c33b87f8-1004-402c-ad4b-83d8a353d834'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.988427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] The token cache is being invalidate because project 1658ed9b107a46e8bd8fd6d3c9978ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:31.988722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.989381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c33b87f8-1004-402c-ad4b-83d8a353d834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:31.991549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1289/2577] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:31 2022] DELETE /identity/v3/projects/1658ed9b107a46e8bd8fd6d3c9978ad4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:31.996459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:31.996754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:31.997037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.019183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:32.020422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:32.084818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1289/2578] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:32.092700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.092993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.093281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.113343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:32.115218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:32.123218 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1290/2579] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:32.127588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.138369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d16f3-3206-4f76-98a9-3b357019526a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.143275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.143961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.144313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.144804 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.145338 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:32.145671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.147418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.159688 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1290/2580] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:32.164475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.173943 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df1af1-b001-4cc9-9468-e4ba7c67c75d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.179741 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.180352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.181039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.181039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.181528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.190872 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.218120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1291/2581] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:32.219559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.230320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be1155-01b6-40e4-b41b-a26eb2c0e1d0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.264637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:32.292403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.293592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.293874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.294157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.294832 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.296782 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.325976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1291/2582] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:32.330090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.342233 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225560ab-6651-4b4e-8924-88d4c1cac1f8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.346920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.347526 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.347816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.348096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.348782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.351399 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.361964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1292/2583] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:32.366583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.377087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bacdec-eb4b-49ae-9374-69d25dbe74f5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.382130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.382768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.383055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.383334 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.383985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.385630 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69387e4-3d88-4a9c-ba36-862177795512 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.398744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1292/2584] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:32.400523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.410359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bfc3b-9e4f-4f81-a9bb-ccb23d6973ba admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.416695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.417357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.417642 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.417921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] PATH_INFO: `/v3/projects/1f19ced9b011463fa3a7826bdd9f7c16/users/3780e41ada1e4250bdd003fc2da240ae/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.432913 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f19ced9b011463fa3a7826bdd9f7c16, user_id=3780e41ada1e4250bdd003fc2da240ae, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.435639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.450072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1293/2585] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:32 2022] PUT /identity/v3/projects/1f19ced9b011463fa3a7826bdd9f7c16/users/3780e41ada1e4250bdd003fc2da240ae/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:32.461168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.461464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.461748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.492302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] MFA Rules not processed for user `3780e41ada1e4250bdd003fc2da240ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:32.493783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:32.593102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1293/2586] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:32.594494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.678889 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5acd-1a9a-4957-8c5b-517837e1cc96 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.685010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.685176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.685425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.685897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.686472 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.688680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.729293 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1294/2587] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:32.730539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.744913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e23eb-990a-4b7c-9926-7ba48ca653a2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.828534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.829446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.829806 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.830159 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.831024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.833813 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.870450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1294/2588] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:32 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:32.871269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.883051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e85e4e-571a-4d91-b9cd-97922d3b29d6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.888742 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.889383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.889708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.889988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.890680 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.892364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.905050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1295/2589] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:32.909890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.921747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523c0f5-a249-43ce-a8f6-3ee8f99a7434 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:32.933195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:32.934010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:32.934403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:32.934932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] PATH_INFO: `/v3/projects/9c7ff5720bdb49d697f41280622258d2/users/78d0e47835ad414db6e28ab2bd5fced1/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:32.955673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7ff5720bdb49d697f41280622258d2, user_id=78d0e47835ad414db6e28ab2bd5fced1, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:32.959125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:32.976396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1295/2590] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:32 2022] PUT /identity/v3/projects/9c7ff5720bdb49d697f41280622258d2/users/78d0e47835ad414db6e28ab2bd5fced1/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:32.979896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:32.990303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb24c-d7a9-4e39-ba39-f3601a2d4f42 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:33.050157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:33.050809 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.051102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.051369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.052071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:33.053651 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02847e9a-8caf-4349-9872-8ba771d072d7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:33.054968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:33.066532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1296/2591] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:33.070609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:33.081058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e4a12-8280-4753-8d1a-824925cc4af4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:33.086304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:33.086982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.087258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.087530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] PATH_INFO: `/v3/projects/9c7ff5720bdb49d697f41280622258d2/users/78d0e47835ad414db6e28ab2bd5fced1/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.089428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7ff5720bdb49d697f41280622258d2, user_id=78d0e47835ad414db6e28ab2bd5fced1, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:33.092096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:33.095147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:33.105175 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1296/2592] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:33 2022] PUT /identity/v3/projects/9c7ff5720bdb49d697f41280622258d2/users/78d0e47835ad414db6e28ab2bd5fced1/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:33.116310 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.116599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.116886 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.145463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] MFA Rules not processed for user `78d0e47835ad414db6e28ab2bd5fced1`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:33.147766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:33.235110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1297/2593] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:33.248797 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:33.250727 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:33.325379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde14d23-e727-418f-8bc6-325caca9f8d7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:33.330412 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:33.331068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.331347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.331621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.342669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:33.344333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:33.355509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1297/2594] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:33.431756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:33.441950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34172c69-b352-4747-b9bb-1855f85d80e6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:33.446529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:33.447245 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.447437 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.447712 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.509838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:33.511337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad71cb-76d0-4c3a-bcb5-12c115e7b5c8 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:33.520535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1298/2595] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:33.793512 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:33.860063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed254b-1ef6-4e72-8fb9-e739a44d76d1 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:33.865285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:33.865968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.866300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.866523 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] PATH_INFO: `/v3/users/3780e41ada1e4250bdd003fc2da240ae` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.867946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3780e41ada1e4250bdd003fc2da240ae)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:33.869873 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:33.919991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3780e41ada1e4250bdd003fc2da240ae'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:33.920673 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3780e41ada1e4250bdd003fc2da240ae'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:33.928544 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3780e41ada1e4250bdd003fc2da240ae'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:33.929217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3780e41ada1e4250bdd003fc2da240ae'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:33.937217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3780e41ada1e4250bdd003fc2da240ae'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:33.958501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1298/2596] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:33 2022] DELETE /identity/v3/users/3780e41ada1e4250bdd003fc2da240ae => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:33.964327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874808-c45c-479c-8cb7-687b537a525c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:33.964846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874808-c45c-479c-8cb7-687b537a525c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:33.964947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874808-c45c-479c-8cb7-687b537a525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:33.984803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-36874808-c45c-479c-8cb7-687b537a525c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:33.985994 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-36874808-c45c-479c-8cb7-687b537a525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.053965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1299/2597] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:34.064599 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:34.066021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.086904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:34.132109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0c57-7364-4311-abbc-523c28d2ff0e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:34.135850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:34.136473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.136744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.137015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.146807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:34.148238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f035686-87c8-4413-96bb-9d37f8c72e3b service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:34.157902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1299/2598] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:34.477961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36874808-c45c-479c-8cb7-687b537a525c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:34.479593 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-36874808-c45c-479c-8cb7-687b537a525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.493963 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36874808-c45c-479c-8cb7-687b537a525c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:34.499355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:34.500215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.500499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.501069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] PATH_INFO: `/v3/projects/1f19ced9b011463fa3a7826bdd9f7c16` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.502425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f19ced9b011463fa3a7826bdd9f7c16)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:34.516162 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:34.547679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:34.552747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f9a0-1b72-4c4d-815a-7418ea1db63e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.553307 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f9a0-1b72-4c4d-815a-7418ea1db63e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.553397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8f9a0-1b72-4c4d-815a-7418ea1db63e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.554753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1300/2599] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:44:34.562904 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.563277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.563625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.587739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:34.589386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.593944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f19ced9b011463fa3a7826bdd9f7c16', 'request_id': 'req-90deebf8-7eaf-4213-a559-648bcfafdc9e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.601482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f19ced9b011463fa3a7826bdd9f7c16', 'request_id': 'req-90deebf8-7eaf-4213-a559-648bcfafdc9e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.609112 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f19ced9b011463fa3a7826bdd9f7c16', 'request_id': 'req-90deebf8-7eaf-4213-a559-648bcfafdc9e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.625942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] The token cache is being invalidate because project 1f19ced9b011463fa3a7826bdd9f7c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:34.626232 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.626970 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.632280 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1300/2600] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:34 2022] DELETE /identity/v3/projects/1f19ced9b011463fa3a7826bdd9f7c16 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:34.633080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:34.634500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.672202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1301/2601] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:44:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:34.684430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:34.685835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.720234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deebf8-7eaf-4213-a559-648bcfafdc9e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:34.725258 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:34.726015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.726303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.726999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] PATH_INFO: `/v3/users/78d0e47835ad414db6e28ab2bd5fced1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.728222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d0e47835ad414db6e28ab2bd5fced1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:34.730192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:34.754773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fde38-7130-4e04-b901-67afd2e44d4c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:34.759155 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:34.759763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.760055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.760333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.771738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:34.773191 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c17e1-855c-44bc-9868-74f73c9d446c service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:34.784010 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1302/2602] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:34.787219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d0e47835ad414db6e28ab2bd5fced1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.787969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d0e47835ad414db6e28ab2bd5fced1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.788659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d0e47835ad414db6e28ab2bd5fced1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.796768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d0e47835ad414db6e28ab2bd5fced1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.809972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c064e913-44c6-4a67-a49e-ab50c157c466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d0e47835ad414db6e28ab2bd5fced1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:34.823616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1301/2603] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:34 2022] DELETE /identity/v3/users/78d0e47835ad414db6e28ab2bd5fced1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:34.861143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.861704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.861972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.863459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1303/2604] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:44:34.869201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4283c2-fe17-4d9c-9f2e-c21a5fc635dc None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.869490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4283c2-fe17-4d9c-9f2e-c21a5fc635dc None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.869765 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4283c2-fe17-4d9c-9f2e-c21a5fc635dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.890353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-da4283c2-fe17-4d9c-9f2e-c21a5fc635dc None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:34.891588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-da4283c2-fe17-4d9c-9f2e-c21a5fc635dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.910141 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:34.953477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1302/2605] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:44:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:34.965915 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:34.967667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:34.982432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9c57c-efe2-4dbf-b13f-15ff43d1de8d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:34.986089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:34.986811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:34.987001 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:34.987260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:34.996042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:34.997543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243f383-a8ce-4d23-b694-952d1bb34cfe service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.006823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1304/2606] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:35.111335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.111897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.111897 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.113422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1303/2607] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:44:35.118982 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.119275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.119583 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.139385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:35.140778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:35.157581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.159339 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:35.186753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:35.217211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1305/2608] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:35.229205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.230292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14d7f-9fd0-4d7f-a371-029aab7ab0c6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.234834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.235463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.235738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.236007 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] PATH_INFO: `/v3/projects/9c7ff5720bdb49d697f41280622258d2` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.237139 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c7ff5720bdb49d697f41280622258d2)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.238563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750684d-cc0c-442d-99ce-f91539c06685 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.239073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.243157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.244088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.244373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.244648 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.256055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.257494 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92849f28-83b5-4e21-b45e-bed63c5c6574 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.268750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1306/2609] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:35.286198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c7ff5720bdb49d697f41280622258d2', 'request_id': 'req-44d41b3b-6832-42c4-ae95-135a2a6d293b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:35.291745 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c7ff5720bdb49d697f41280622258d2', 'request_id': 'req-44d41b3b-6832-42c4-ae95-135a2a6d293b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:35.297951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c7ff5720bdb49d697f41280622258d2', 'request_id': 'req-44d41b3b-6832-42c4-ae95-135a2a6d293b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:35.311480 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] The token cache is being invalidate because project 9c7ff5720bdb49d697f41280622258d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:35.311594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:35.312178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-44d41b3b-6832-42c4-ae95-135a2a6d293b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:35.314537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1304/2610] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:35 2022] DELETE /identity/v3/projects/9c7ff5720bdb49d697f41280622258d2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:35.320905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.321216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.321479 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.355610 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:35.357524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:35.443034 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1307/2611] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:35.445604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.445893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.446170 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.467683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:35.468860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:35.483894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1305/2612] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:35.488173 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.501434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de6a75-91f0-45f7-9353-e40460af5aab None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.507758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.508373 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.508655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.508933 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.509591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:35.509898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.511534 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.529244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1308/2613] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:35.534029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.547452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4140aed-3290-4a58-9616-20d901dd2acb None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.561560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.562313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.562685 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.563291 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.563715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.565686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.630887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1306/2614] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:35.631329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.643008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f9171-e02c-4575-8428-4578e8888478 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.718851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.719436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.719721 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.719994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.720633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.722772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.749762 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1309/2615] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:35.753944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.767751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88f6cf-779b-4f57-8613-aacfc186766e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.774136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.774788 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.775074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.775355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.776009 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.777649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.790382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1307/2616] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:35.794565 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.809146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdd533-a02e-49cf-b1f1-2f9e6f22da17 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.814526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.815634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.815979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.816626 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.816867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.818790 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650726ce-c716-4b14-a3a5-fd11fc8b3f7f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.831625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1310/2617] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:35.837580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:35.852564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db37bb-3fd4-4644-b2a5-01507ad22e47 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:35.859540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:35.860211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.860504 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.860907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] PATH_INFO: `/v3/projects/5e38428e428a4f07afd1f12c063a67a9/users/7e3929e5edfc465fabee3ba9bf3874d5/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.887663 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e38428e428a4f07afd1f12c063a67a9, user_id=7e3929e5edfc465fabee3ba9bf3874d5, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:35.890435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:35.901194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:35.928750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1308/2618] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:35 2022] PUT /identity/v3/projects/5e38428e428a4f07afd1f12c063a67a9/users/7e3929e5edfc465fabee3ba9bf3874d5/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:35.931260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:35.931547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:35.931876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:35.969009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] MFA Rules not processed for user `7e3929e5edfc465fabee3ba9bf3874d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:35.970367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:35.988445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:36.073711 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1311/2619] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:36.077775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.145113 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963223ee-218e-48d8-a3e5-9bd2cf538464 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.150178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.150847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.151133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.151409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.152086 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.154053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.187968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1309/2620] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:36.191988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.202609 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aae4d-729d-46be-b1ec-b78adadb113c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.260177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.260779 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.261061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.261336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.261972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.263917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.284459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1312/2621] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:36.289110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.299404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51d779-c560-4399-bc12-e3d5e217f153 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.305038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.305641 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.305925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.306205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.306895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.308484 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.319516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1310/2622] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:36.323647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.333792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f536-3ac4-4311-9374-baa6a904e93d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.338748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.339388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.339629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.339951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.340679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.342417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91f786-cb3d-46f3-a96a-a76a36c389b4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.354292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1313/2623] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:36.358847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.369787 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9bdc-cb5b-4c20-b308-18610e9d74fb admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.374492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.375176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.375458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.375730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] PATH_INFO: `/v3/projects/358b912924ac45e8b082a04607091f11/users/e61376ce10fc4dba81e09cf285205e8b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.391536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=358b912924ac45e8b082a04607091f11, user_id=e61376ce10fc4dba81e09cf285205e8b, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.394242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.407717 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1311/2624] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:36 2022] PUT /identity/v3/projects/358b912924ac45e8b082a04607091f11/users/e61376ce10fc4dba81e09cf285205e8b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:36.420088 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.420379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.420659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.451143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] MFA Rules not processed for user `e61376ce10fc4dba81e09cf285205e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:36.452344 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:36.521612 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1314/2625] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:36.526357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.594617 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f4ec3-0b23-418f-ac89-301bdfa09165 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.599635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.600229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.600676 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.600842 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.601829 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.603752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.632508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1312/2626] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:36.635600 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.646691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f1cdf-bc0f-41f9-af8d-0b0fb652e4ce None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.666998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:36.708153 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.708883 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.709300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.709696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.710545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.712816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.738654 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1315/2627] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:36.743749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.758513 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255212-ca72-4970-aadc-21a1e0b717f2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.763133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.763728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.764004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.764273 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.764910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.766477 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.777082 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1313/2628] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:36.781250 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.791339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b13b2-2181-4132-a661-8927bf135dff admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.795902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.796543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.796858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.797133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] PATH_INFO: `/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f/users/6c742c078b984a5699afbc478fed5b66/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.812296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e50398e43b49fdb3647e64e9b9e89f, user_id=6c742c078b984a5699afbc478fed5b66, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.814973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.828521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1316/2629] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:36 2022] PUT /identity/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f/users/6c742c078b984a5699afbc478fed5b66/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:36.832433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.842785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de3560-9049-4268-8de0-fb198397d305 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.861337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:36.902887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.903499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.903781 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.904059 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.904699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.906289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d074d9-9b22-4152-8f3c-b44ce861cd51 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.917032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1314/2630] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:36.921376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:36.932453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815695-a93d-4b49-9bcc-7a048642f17c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:36.937114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:36.937948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.938363 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.938773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] PATH_INFO: `/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f/users/6c742c078b984a5699afbc478fed5b66/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.941328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e50398e43b49fdb3647e64e9b9e89f, user_id=6c742c078b984a5699afbc478fed5b66, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:36.945073 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:36.955414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1317/2631] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:36 2022] PUT /identity/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f/users/6c742c078b984a5699afbc478fed5b66/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:36.966355 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dbfe1-42a4-47db-a493-246d97dd242f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:36.966692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dbfe1-42a4-47db-a493-246d97dd242f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:36.966979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dbfe1-42a4-47db-a493-246d97dd242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:36.997649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-594dbfe1-42a4-47db-a493-246d97dd242f None None] MFA Rules not processed for user `6c742c078b984a5699afbc478fed5b66`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:36.998990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-594dbfe1-42a4-47db-a493-246d97dd242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:37.093116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1315/2632] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:37.110417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:37.112424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:37.184724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2077c9-5416-41d7-bb42-d9140723c6c6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:37.189596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:37.190280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:37.190916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:37.191223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:37.264120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:37.267258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:37.280211 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1318/2633] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:40.332555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:40.333591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:40.333874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:40.353568 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:40.354803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:40.425450 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1316/2634] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:44:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:40.438457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:40.440082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:40.477981 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:40.519078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeec93-63f0-4138-8649-56d2c830686e service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:40.524720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:40.525328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:40.525607 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:40.525878 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:40.534617 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:40.536414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:40.544993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1319/2635] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:44:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:44:41.035805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:41.048035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf0c7d-eac8-475d-955c-bdd88c6f2ee1 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:41.054404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:41.055056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:41.055341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:41.055615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:41.114612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:41.128686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:41.130951 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:41.151032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1317/2636] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:41.807918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:41.818058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:41.831900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de39405-7a3f-4bdb-87ab-cef32a1e5c5b service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:41.847567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:41.848365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:41.848645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:41.848997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:41.859957 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:41.861906 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db53c4e-d0ca-4fda-b07a-1bf73eb8d2f3 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:41.874365 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1320/2637] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:45.241464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:45.243039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:45.335292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8649b-47ca-4c6c-a122-df9e5a875e08 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:45.339864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:45.340443 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:45.340723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:45.341079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] PATH_INFO: `/v3/users/9d6d7f575304448a89d06e25ccd2d119` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:45.342243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d6d7f575304448a89d06e25ccd2d119)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:45.344235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:45.401262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d6d7f575304448a89d06e25ccd2d119'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:45.401922 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d6d7f575304448a89d06e25ccd2d119'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:45.402554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d6d7f575304448a89d06e25ccd2d119'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:45.411540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d6d7f575304448a89d06e25ccd2d119'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:45.425959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d6d7f575304448a89d06e25ccd2d119'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:45.447155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1318/2638] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:45 2022] DELETE /identity/v3/users/9d6d7f575304448a89d06e25ccd2d119 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:45.451760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:45.452375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:45.452529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:45.475433 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:45.476929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:45.546663 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1321/2639] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:45.557140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:45.558556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:45.627147 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c1156-b86a-4eef-969c-039a5c792964 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:45.631850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:45.632545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:45.632914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:45.633257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:45.644095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:45.645941 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:45.661640 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1319/2640] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:46.084763 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.087679 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.108502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4241-eb8a-4520-b5d5-9fb73089a53e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.132337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.132337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.132337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.132337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] PATH_INFO: `/v3/projects/0ce10007c6874c1b92aa348c682fc043` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.134068 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce10007c6874c1b92aa348c682fc043)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.136158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.183111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.186413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.209053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab6fd8-0e71-4115-b7c6-550a5f47a46b service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.217195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:46.241816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce10007c6874c1b92aa348c682fc043', 'request_id': 'req-866dd3e8-e447-49f3-aa65-18e956c703eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.256701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce10007c6874c1b92aa348c682fc043', 'request_id': 'req-866dd3e8-e447-49f3-aa65-18e956c703eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.265015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce10007c6874c1b92aa348c682fc043', 'request_id': 'req-866dd3e8-e447-49f3-aa65-18e956c703eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.295488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] The token cache is being invalidate because project 0ce10007c6874c1b92aa348c682fc043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:46.296296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.298118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-866dd3e8-e447-49f3-aa65-18e956c703eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.299887 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1322/2641] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:46 2022] DELETE /identity/v3/projects/0ce10007c6874c1b92aa348c682fc043 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:46.305694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5500c8-bc59-4f63-8d94-4967468929e5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.306103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5500c8-bc59-4f63-8d94-4967468929e5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.306416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5500c8-bc59-4f63-8d94-4967468929e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.307942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.308565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.308846 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.309121 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] PATH_INFO: `/v3/users/7e3929e5edfc465fabee3ba9bf3874d5` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.310335 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3929e5edfc465fabee3ba9bf3874d5)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.312766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.350748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9b5500c8-bc59-4f63-8d94-4967468929e5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:46.355106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9b5500c8-bc59-4f63-8d94-4967468929e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.374684 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1323/2642] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:46.381595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.381964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.382302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.415822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:46.417194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.432928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1324/2643] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:46.434051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.446826 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee99b4c-3dc7-42b4-ae83-b68a001166cf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.452752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.454184 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.454661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.455111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.455945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:46.456438 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.458322 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.473879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1325/2644] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:46.485525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.493732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:46.497908 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f68ca-50dc-48db-a971-ff2901566a16 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.524676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.525633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.526097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.526618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.527618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.532637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3929e5edfc465fabee3ba9bf3874d5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.533581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3929e5edfc465fabee3ba9bf3874d5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.534351 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3929e5edfc465fabee3ba9bf3874d5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.537840 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.546192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3929e5edfc465fabee3ba9bf3874d5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.565072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-8dd27e66-2737-4b3d-918f-829c83a53ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3929e5edfc465fabee3ba9bf3874d5'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:46.580194 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1326/2645] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:46.585341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.587289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.589608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1320/2646] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:46 2022] DELETE /identity/v3/users/7e3929e5edfc465fabee3ba9bf3874d5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:46.594227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.594536 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.594818 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.620461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:46.621887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.698417 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e1cfb-9c39-4bb2-b70e-b4c6c44c3d46 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.703294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.704065 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.704418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.704765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.705471 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.705921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1321/2647] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:46.708504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.717110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.719887 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:46.737161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1327/2648] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:46.741481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.754072 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25366e90-62f5-4084-94cd-1d1d694a1388 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.759219 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:46.761272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.761961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.762402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.762830 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.763584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.766023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.782086 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1328/2649] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:46.786413 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.788632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7641f-6e6a-4ded-8834-a6e52211888e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.792606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.793326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.793671 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.794010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.797644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598c212-165b-4acb-81f4-7c7ca49c7182 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.805431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.807117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.808326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.809217 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.809710 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.810204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.811174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.814029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7ff8-bb5c-44dc-9868-7cfba9ad3218 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.827958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1322/2650] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:46.829106 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1329/2651] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:44:46.834272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:46.845425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4e8cb-679b-4186-a9f9-6c92c455c5c8 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:46.851953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:46.852773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.853174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.853508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] PATH_INFO: `/v3/projects/163a3c48bbb34d879d5a2f5b267f6837/users/1dee1c72e28a4fc9867b472a26202158/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.871767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=163a3c48bbb34d879d5a2f5b267f6837, user_id=1dee1c72e28a4fc9867b472a26202158, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:46.875094 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:46.903073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1323/2652] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:46 2022] PUT /identity/v3/projects/163a3c48bbb34d879d5a2f5b267f6837/users/1dee1c72e28a4fc9867b472a26202158/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:46.905849 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:46.906198 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:46.906519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:46.947171 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] MFA Rules not processed for user `1dee1c72e28a4fc9867b472a26202158`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:46.948532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.111884 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1330/2653] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:47.130256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:47.133238 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.237316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d173d-6c79-4739-8f2e-4d826c344b9c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:47.247209 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:47.248984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.250083 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.251168 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.271527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:47.273189 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:47.290905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1324/2654] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:47.311986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:47.317816 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.329673 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:47.382312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:47.408790 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ead0d-d154-4047-94ce-bc78f8f58ffc service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:47.418047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:47.418047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.418047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.418490 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708e160-f6c4-4e81-bb56-833c172967a6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:47.418608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] PATH_INFO: `/v3/users/1dee1c72e28a4fc9867b472a26202158` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.420932 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:47.421570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.421823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dee1c72e28a4fc9867b472a26202158)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:47.421934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.422127 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] PATH_INFO: `/v3/projects/5e38428e428a4f07afd1f12c063a67a9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.423407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e38428e428a4f07afd1f12c063a67a9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:47.423909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:47.425339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:47.506326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e38428e428a4f07afd1f12c063a67a9', 'request_id': 'req-b50a15fd-ead1-4f05-9699-2053710a7d9b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.519500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e38428e428a4f07afd1f12c063a67a9', 'request_id': 'req-b50a15fd-ead1-4f05-9699-2053710a7d9b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.527502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e38428e428a4f07afd1f12c063a67a9', 'request_id': 'req-b50a15fd-ead1-4f05-9699-2053710a7d9b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.529408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dee1c72e28a4fc9867b472a26202158'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.530117 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dee1c72e28a4fc9867b472a26202158'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.534969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dee1c72e28a4fc9867b472a26202158'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.549898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] The token cache is being invalidate because project 5e38428e428a4f07afd1f12c063a67a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:47.550356 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.552488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.553691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:47.556968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1331/2655] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:47 2022] DELETE /identity/v3/projects/5e38428e428a4f07afd1f12c063a67a9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:47.563030 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dee1c72e28a4fc9867b472a26202158'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.563235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:47.564280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.589363 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f4df4854-c8c5-472b-b19e-acb143d61eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dee1c72e28a4fc9867b472a26202158'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.613276 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1325/2656] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:47 2022] DELETE /identity/v3/users/1dee1c72e28a4fc9867b472a26202158 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:47.628714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.632489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.632874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.643229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a15fd-ead1-4f05-9699-2053710a7d9b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:47.653705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:47.654958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.737107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:47.741729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.741880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1326/2657] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:47.741974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.741974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] PATH_INFO: `/v3/users/e61376ce10fc4dba81e09cf285205e8b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.742556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61376ce10fc4dba81e09cf285205e8b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:47.744666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:47.749833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:47.754783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.785225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:47.824064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61376ce10fc4dba81e09cf285205e8b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.824856 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61376ce10fc4dba81e09cf285205e8b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.834225 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61376ce10fc4dba81e09cf285205e8b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.834999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61376ce10fc4dba81e09cf285205e8b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.844897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61376ce10fc4dba81e09cf285205e8b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:47.855703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff1292-c91e-409d-a4b7-49121d2f328e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:47.860308 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:47.860932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:47.861457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:47.861805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:47.864919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:47.876586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1332/2658] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:47 2022] DELETE /identity/v3/users/e61376ce10fc4dba81e09cf285205e8b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:47.886865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:47.890337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:47.901801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1327/2659] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:48.320457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.323047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.408885 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.410307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.423588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a3692-612d-4814-aa77-c64eaadc2aaf service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.429045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023871-b677-4655-a867-294ce42f442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.429593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e52eea-36e7-4783-a7f0-b0ddef864fc8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.429734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023871-b677-4655-a867-294ce42f442e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.429949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023871-b677-4655-a867-294ce42f442e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.430224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023871-b677-4655-a867-294ce42f442e admin admin] PATH_INFO: `/v3/projects/358b912924ac45e8b082a04607091f11` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.431498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69023871-b677-4655-a867-294ce42f442e admin admin] RBAC: Authorizing `identity:delete_project(project_id=358b912924ac45e8b082a04607091f11)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.433409 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69023871-b677-4655-a867-294ce42f442e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.435852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.436669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.436985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.437362 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] PATH_INFO: `/v3/projects/163a3c48bbb34d879d5a2f5b267f6837` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.438613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163a3c48bbb34d879d5a2f5b267f6837)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.440647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.487986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358b912924ac45e8b082a04607091f11', 'request_id': 'req-69023871-b677-4655-a867-294ce42f442e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.492457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163a3c48bbb34d879d5a2f5b267f6837', 'request_id': 'req-e2b44a79-baa4-4921-b4e8-719ce85ead42'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.494629 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358b912924ac45e8b082a04607091f11', 'request_id': 'req-69023871-b677-4655-a867-294ce42f442e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.498601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163a3c48bbb34d879d5a2f5b267f6837', 'request_id': 'req-e2b44a79-baa4-4921-b4e8-719ce85ead42'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.500921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358b912924ac45e8b082a04607091f11', 'request_id': 'req-69023871-b677-4655-a867-294ce42f442e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.504835 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:48.504995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163a3c48bbb34d879d5a2f5b267f6837', 'request_id': 'req-e2b44a79-baa4-4921-b4e8-719ce85ead42'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.517471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] The token cache is being invalidate because project 358b912924ac45e8b082a04607091f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:48.517767 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.518475 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.520227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1328/2660] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:48 2022] DELETE /identity/v3/projects/358b912924ac45e8b082a04607091f11 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:48.520729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] The token cache is being invalidate because project 163a3c48bbb34d879d5a2f5b267f6837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:48.521224 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.521696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2b44a79-baa4-4921-b4e8-719ce85ead42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.524010 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.526929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1333/2661] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:48 2022] DELETE /identity/v3/projects/163a3c48bbb34d879d5a2f5b267f6837 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:48.527816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.528227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed630e-9a25-406e-ba2a-76d1a2859f57 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.528517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed630e-9a25-406e-ba2a-76d1a2859f57 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.528799 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed630e-9a25-406e-ba2a-76d1a2859f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.550012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a2ed630e-9a25-406e-ba2a-76d1a2859f57 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:48.551834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a2ed630e-9a25-406e-ba2a-76d1a2859f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.572776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:48.600649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023871-b677-4655-a867-294ce42f442e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.616484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1334/2662] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:48.624259 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.624545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.624823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.645427 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:48.646660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.655739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1335/2663] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:48.659477 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.668874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.669252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203ed7b-a277-43b1-8106-97050f9eb13a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.669509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.669789 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.670067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] PATH_INFO: `/v3/users/6c742c078b984a5699afbc478fed5b66` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.671214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c742c078b984a5699afbc478fed5b66)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.673130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.673928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.674518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.674838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.675113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.675793 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:48.676197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.677728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.689465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1336/2664] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:44:48.694128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.703946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d338-23a4-43f7-8f98-a9beda534e76 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.709920 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.710541 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.710847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.711121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.711744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.713644 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.722287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c742c078b984a5699afbc478fed5b66'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.723546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c742c078b984a5699afbc478fed5b66'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.724217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c742c078b984a5699afbc478fed5b66'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.731960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c742c078b984a5699afbc478fed5b66'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.742489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1337/2665] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:48 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:48.745100 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c742c078b984a5699afbc478fed5b66'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:48.746601 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.748243 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:48.757378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1329/2666] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:48 2022] DELETE /identity/v3/users/6c742c078b984a5699afbc478fed5b66 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:48.815652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d0c0c-4845-47f2-9f0a-dbc60b53dca4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.875228 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.875993 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.876345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.876618 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.877240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.879367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.905144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1338/2667] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:48 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:48.906300 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.917971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b6f6a-041a-471f-abd1-22be165d9ed4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.923343 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.924235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.924555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.924840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.925974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.927925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.940716 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1330/2668] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:48.946491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.956531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482d063-16b7-46cf-a0f9-f94ea863167e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.961972 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.962582 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.962899 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.963313 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:48.963971 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:48.965562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d332b-53a3-4175-bc03-66ebe1ae1844 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:48.977353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1339/2669] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:48.981583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:48.991646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490950-fc1d-4f71-a339-5bd8036773a4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:48.996482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:48.997142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:48.997417 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:48.997686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] PATH_INFO: `/v3/projects/df59084726864fde8cf6b2f24b7336de/users/5d933777f9134bae90720348d29d9bf1/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.008127 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:49.018191 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df59084726864fde8cf6b2f24b7336de, user_id=5d933777f9134bae90720348d29d9bf1, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.022014 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.041662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1331/2670] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:48 2022] PUT /identity/v3/projects/df59084726864fde8cf6b2f24b7336de/users/5d933777f9134bae90720348d29d9bf1/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:49.053182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.053736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.054370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.062324 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.064231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.084732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:49.100805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] MFA Rules not processed for user `5d933777f9134bae90720348d29d9bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:49.102509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.142904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244387-3bc6-4814-9bdb-6d5540a19f89 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.148430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.149079 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.149349 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.149626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] PATH_INFO: `/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.150864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e50398e43b49fdb3647e64e9b9e89f)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.153414 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.179408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1340/2671] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:49.194060 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.195519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.218713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e50398e43b49fdb3647e64e9b9e89f', 'request_id': 'req-e5578888-2c92-4658-8969-c036d039b5f1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:49.225104 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e50398e43b49fdb3647e64e9b9e89f', 'request_id': 'req-e5578888-2c92-4658-8969-c036d039b5f1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:49.232376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e50398e43b49fdb3647e64e9b9e89f', 'request_id': 'req-e5578888-2c92-4658-8969-c036d039b5f1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:49.250560 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] The token cache is being invalidate because project d9e50398e43b49fdb3647e64e9b9e89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:49.250904 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:49.251571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e5578888-2c92-4658-8969-c036d039b5f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:49.253090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1332/2672] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:49 2022] DELETE /identity/v3/projects/d9e50398e43b49fdb3647e64e9b9e89f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:49.258893 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fb9c-a883-4b3f-9201-467d36827b01 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.259176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fb9c-a883-4b3f-9201-467d36827b01 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.259468 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614fb9c-a883-4b3f-9201-467d36827b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.276765 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0db0ce-3258-4746-8c4b-70b7b6b0014e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.288550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f614fb9c-a883-4b3f-9201-467d36827b01 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:49.289775 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f614fb9c-a883-4b3f-9201-467d36827b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.331704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.332498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.332770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.333043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.335024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.368295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1333/2673] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:49.424287 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.425215 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.437220 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1341/2674] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:49.438411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.438926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.439023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.463784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:49.465011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:49.475719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1334/2675] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:49.480077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.493638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d85df-b049-46f9-85c8-f88d90093594 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.498497 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.499149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.499613 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.499905 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.500570 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:49.500879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.502556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.510167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:49.522183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1342/2676] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:49.523727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.534589 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00064a6-e318-4b2a-974f-1a6491b97e0d None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.542336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.543198 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.543917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.544609 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.546175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.553705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.563834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:49.591859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1335/2677] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:49.593166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.609703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df468c-5a84-4de4-afec-cb407daa09b8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.694796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.695502 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.695862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.696208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.696918 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.698978 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.722403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1343/2678] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:49.728956 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.741626 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed6a64-a44c-4354-8c39-67efedf58bde admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.746290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.747037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.747317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.747583 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.748320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.750039 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.764165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1336/2679] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:49.770896 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.781717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004bcb-7e68-44e9-892b-3f5243aaef6f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.787102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.787787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.788140 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.788484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.789207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.791525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775713c-0820-4cff-abdd-663226b3724e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.804962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1344/2680] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:49.810381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:49.826237 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6608b-2bae-4839-9baf-e6a8ba9cadf5 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:49.833222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:49.837660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.837660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.837660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] PATH_INFO: `/v3/projects/84fbeb1fa20849d99218a9e1d467b027/users/99bdbf6076324a3db915e1f3bee26aa2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.850496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84fbeb1fa20849d99218a9e1d467b027, user_id=99bdbf6076324a3db915e1f3bee26aa2, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:49.855353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:49.871706 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1337/2681] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:49 2022] PUT /identity/v3/projects/84fbeb1fa20849d99218a9e1d467b027/users/99bdbf6076324a3db915e1f3bee26aa2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:49.885573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:49.886120 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:49.886525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:49.926328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] MFA Rules not processed for user `99bdbf6076324a3db915e1f3bee26aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:49.928015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:50.024221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1345/2682] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:50.028876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.099710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355deeb-1d16-4803-8416-38d5035c8f59 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.104754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.105352 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.105627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.105896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.106544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.108447 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.135619 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1338/2683] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:50.139911 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.150760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202e83a-7cca-4995-aadc-32222dc5b023 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.208578 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.209079 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.209361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.209640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.210268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.212226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.223059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:50.237750 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1346/2684] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:50.240694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.255531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119741-a750-4aa5-a821-6b2860096cce admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.261158 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.261766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.262051 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.262328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.263005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.264616 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.279241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1339/2685] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:50.283301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.297773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1930db-9059-4ac9-9eae-31b92fe456a8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.303589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.304199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.304482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.304754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.305396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.307045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2662d-9ab1-479a-a122-d2e4c06b4419 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.318432 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1347/2686] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:50.322628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.333421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1ec0-68cf-4573-b5fc-52abb60b545a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.338129 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.338813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.339093 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.339368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] PATH_INFO: `/v3/projects/eea64264a48e495d8b727d21e74080c1/users/43d735c721e24fbcbd7e2eef8261b004/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.355415 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea64264a48e495d8b727d21e74080c1, user_id=43d735c721e24fbcbd7e2eef8261b004, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.358523 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.372071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1340/2687] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:50 2022] PUT /identity/v3/projects/eea64264a48e495d8b727d21e74080c1/users/43d735c721e24fbcbd7e2eef8261b004/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:50.382888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.383187 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.383467 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.413350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] MFA Rules not processed for user `43d735c721e24fbcbd7e2eef8261b004`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:50.414575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:50.479077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1348/2688] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:50.483750 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.545214 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f7ec-316d-4bdb-8ad8-42ad363889ce admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.549859 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.550456 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.550778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.551269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.552001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.553888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.563235 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:50.583435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1341/2689] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:50.587628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.598260 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2e159-ff1f-44c0-b526-90a216386acb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.654606 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.655235 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.655519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.655796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.656430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.658361 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.683466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1349/2690] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:50.684824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.696074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7147-bd58-470d-a1eb-378bc76baf63 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.700653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.701234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.701506 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.701778 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.702411 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.704025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.714832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1342/2691] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:50.719047 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.729357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a93c-95f3-4966-b821-5a7ceb63123e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.734008 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.734756 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.735036 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.735305 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] PATH_INFO: `/v3/projects/e95820335aef46a6a196c21938477174/users/01936119976b4902b5e8a1629da1efc0/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.750369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95820335aef46a6a196c21938477174, user_id=01936119976b4902b5e8a1629da1efc0, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.753051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.767483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1350/2692] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:50 2022] PUT /identity/v3/projects/e95820335aef46a6a196c21938477174/users/01936119976b4902b5e8a1629da1efc0/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:50.771966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.782759 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7015a1-6f31-4d4d-8f34-1944b3c39c93 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.840187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.840828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.841111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.841388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.842060 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.843714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68849049-edc5-4a3a-bb2e-5209569a1987 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.856981 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1343/2693] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:50.858111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:50.869099 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b372-59ad-484e-98b3-02d6d1ada15a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:50.874029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:50.874704 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.874989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.875266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] PATH_INFO: `/v3/projects/e95820335aef46a6a196c21938477174/users/01936119976b4902b5e8a1629da1efc0/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.877166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95820335aef46a6a196c21938477174, user_id=01936119976b4902b5e8a1629da1efc0, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:50.880540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:50.890869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1351/2694] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:50 2022] PUT /identity/v3/projects/e95820335aef46a6a196c21938477174/users/01936119976b4902b5e8a1629da1efc0/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:44:50.902421 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:50.902751 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:50.903031 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:50.930400 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] MFA Rules not processed for user `01936119976b4902b5e8a1629da1efc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:50.931635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:51.000561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1344/2695] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:51.012551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:51.014281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:51.049770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:51.079966 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f627f-93e1-488f-9b7d-a66c8772b9b8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:51.083984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:51.084588 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:51.084870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:51.085147 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:51.146746 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:51.148316 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:51.157395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1352/2696] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:54.772749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:54.782686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd606dc-4d99-4cc1-b9ea-673f09f7984f None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:54.786940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:54.787540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:54.787824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:54.788099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:54.843305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:54.844693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:54.849073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:54.856420 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1345/2697] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:55.386744 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:55.395641 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7174f-fa74-4347-a527-f30e6b3a89fd service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:55.399888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:55.400931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:55.400931 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:55.401309 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:55.412054 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:55.413575 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbf37-d14b-417e-ab7f-90066ca8ffb7 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:55.424017 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1353/2698] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:57.711710 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:57.713107 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:57.778058 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197913d-14c7-405b-9c2b-dc66a8fb42f6 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:57.782532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:57.783182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:57.783465 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:57.783770 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] PATH_INFO: `/v3/users/5d933777f9134bae90720348d29d9bf1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:57.784914 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d933777f9134bae90720348d29d9bf1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:57.786966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:57.839057 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d933777f9134bae90720348d29d9bf1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:57.839687 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d933777f9134bae90720348d29d9bf1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:57.840342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d933777f9134bae90720348d29d9bf1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:57.848888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d933777f9134bae90720348d29d9bf1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:57.861928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d933777f9134bae90720348d29d9bf1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:57.877905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1346/2699] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:57 2022] DELETE /identity/v3/users/5d933777f9134bae90720348d29d9bf1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:57.882571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:57.883075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:57.883190 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:57.904857 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:57.906076 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:57.984455 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1354/2700] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:57.996260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:57.997720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:58.067579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f603fd-f590-440e-8431-e8c7c953e545 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:58.071262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:58.071865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:58.072145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:58.072418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:58.082499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:58.084201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:58.094111 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1347/2701] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:58.400792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:58.404340 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:58.419447 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bed36-a672-4ee7-a290-a7ec0aff9a82 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:58.425208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:58.426783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:58.426783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:58.427366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] PATH_INFO: `/v3/projects/df59084726864fde8cf6b2f24b7336de` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:58.428380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df59084726864fde8cf6b2f24b7336de)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:58.430280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:58.484653 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df59084726864fde8cf6b2f24b7336de', 'request_id': 'req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:58.491853 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df59084726864fde8cf6b2f24b7336de', 'request_id': 'req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:58.498367 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df59084726864fde8cf6b2f24b7336de', 'request_id': 'req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:58.500149 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:58.512647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] The token cache is being invalidate because project df59084726864fde8cf6b2f24b7336de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:58.513006 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:58.513810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d3deafd1-2039-4fd3-a9bd-ed15994a44c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:58.515386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1355/2702] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:58 2022] DELETE /identity/v3/projects/df59084726864fde8cf6b2f24b7336de => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:59.411187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:59.412580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:59.425213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc147c-7ca9-43d0-8e74-33070a1f7ab1 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:59.441996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:59.485242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:59.485958 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:59.486404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:59.486720 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] PATH_INFO: `/v3/users/99bdbf6076324a3db915e1f3bee26aa2` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:59.487949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bdbf6076324a3db915e1f3bee26aa2)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:59.490080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:59.542621 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bdbf6076324a3db915e1f3bee26aa2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:59.543563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bdbf6076324a3db915e1f3bee26aa2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:59.544342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bdbf6076324a3db915e1f3bee26aa2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:59.555317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bdbf6076324a3db915e1f3bee26aa2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:59.569035 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bdbf6076324a3db915e1f3bee26aa2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:59.583722 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1348/2703] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:59 2022] DELETE /identity/v3/users/99bdbf6076324a3db915e1f3bee26aa2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:59.588246 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:59.588828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:59.588828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:59.608772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:59.609979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:59.681124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1356/2704] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:44:59.692602 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:59.694148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:59.751102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c3aa-556f-4a72-a253-1896bff4adfd admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:59.755433 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:59.756032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:59.756312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:59.756585 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:59.765923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:59.767442 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:59.777918 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1349/2705] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:45:00.058537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:00.060349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:00.074992 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a33fb-9443-43be-844b-51a430fed8ac None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:00.079738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:00.080366 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:00.080665 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:00.080935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] PATH_INFO: `/v3/projects/84fbeb1fa20849d99218a9e1d467b027` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:00.082796 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84fbeb1fa20849d99218a9e1d467b027)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:00.084718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:00.137117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84fbeb1fa20849d99218a9e1d467b027', 'request_id': 'req-e4360450-0e9b-43f0-a5dc-d3240e64ea96'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.143113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84fbeb1fa20849d99218a9e1d467b027', 'request_id': 'req-e4360450-0e9b-43f0-a5dc-d3240e64ea96'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.149921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84fbeb1fa20849d99218a9e1d467b027', 'request_id': 'req-e4360450-0e9b-43f0-a5dc-d3240e64ea96'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.163085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] The token cache is being invalidate because project 84fbeb1fa20849d99218a9e1d467b027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:45:00.163370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.164005 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.165579 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1357/2706] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:45:00 2022] DELETE /identity/v3/projects/84fbeb1fa20849d99218a9e1d467b027 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:45:00.169900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:00.179972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a257a85-a559-486f-b1fc-f79a62553c56 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:00.225179 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:00.246315 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:00.246985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:00.247267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:00.247546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] PATH_INFO: `/v3/users/43d735c721e24fbcbd7e2eef8261b004` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:00.248702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d735c721e24fbcbd7e2eef8261b004)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:00.250857 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:00.298603 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d735c721e24fbcbd7e2eef8261b004'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.299292 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d735c721e24fbcbd7e2eef8261b004'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.299898 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d735c721e24fbcbd7e2eef8261b004'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.308217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d735c721e24fbcbd7e2eef8261b004'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.331044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d735c721e24fbcbd7e2eef8261b004'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.343479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1350/2707] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:45:00 2022] DELETE /identity/v3/users/43d735c721e24fbcbd7e2eef8261b004 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:45:00.627672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:00.629862 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:00.643728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4360450-0e9b-43f0-a5dc-d3240e64ea96 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:00.701133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:00.701760 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:00.702041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:00.702314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] PATH_INFO: `/v3/projects/eea64264a48e495d8b727d21e74080c1` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:00.703459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea64264a48e495d8b727d21e74080c1)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:00.705381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:00.721377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:00.754958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea64264a48e495d8b727d21e74080c1', 'request_id': 'req-be7af72c-c21b-424b-b117-6f0025a17d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.760406 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea64264a48e495d8b727d21e74080c1', 'request_id': 'req-be7af72c-c21b-424b-b117-6f0025a17d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.766465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea64264a48e495d8b727d21e74080c1', 'request_id': 'req-be7af72c-c21b-424b-b117-6f0025a17d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.778917 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] The token cache is being invalidate because project eea64264a48e495d8b727d21e74080c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:45:00.779212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.779844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.781321 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1358/2708] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:45:00 2022] DELETE /identity/v3/projects/eea64264a48e495d8b727d21e74080c1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:45:00.785519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:00.786973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:00.799431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68ae9-ca08-4c82-b931-e66ef6c1b7b6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:00.855646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:00.856264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:00.856542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:00.856813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] PATH_INFO: `/v3/users/01936119976b4902b5e8a1629da1efc0` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:00.857923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01936119976b4902b5e8a1629da1efc0)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:00.860032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:00.907015 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01936119976b4902b5e8a1629da1efc0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.907588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01936119976b4902b5e8a1629da1efc0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.908184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01936119976b4902b5e8a1629da1efc0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.916277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01936119976b4902b5e8a1629da1efc0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.923482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:00.930090 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-50e8ec0d-8181-43bb-b8f4-d02f11fd3dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01936119976b4902b5e8a1629da1efc0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:00.944045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1351/2709] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:45:00 2022] DELETE /identity/v3/users/01936119976b4902b5e8a1629da1efc0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:45:01.216327 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:01.217888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:01.231510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af72c-c21b-424b-b117-6f0025a17d18 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:01.289109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:01.289739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:01.290021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:01.290407 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] PATH_INFO: `/v3/projects/e95820335aef46a6a196c21938477174` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:01.291571 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95820335aef46a6a196c21938477174)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:01.293484 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:01.341691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95820335aef46a6a196c21938477174', 'request_id': 'req-4db82954-ed74-44f9-8e63-cbdf5dd86990'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:01.348041 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95820335aef46a6a196c21938477174', 'request_id': 'req-4db82954-ed74-44f9-8e63-cbdf5dd86990'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:01.354396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95820335aef46a6a196c21938477174', 'request_id': 'req-4db82954-ed74-44f9-8e63-cbdf5dd86990'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:01.368221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] The token cache is being invalidate because project e95820335aef46a6a196c21938477174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:45:01.368567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:01.369210 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:45:01.371267 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1359/2710] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:45:01 2022] DELETE /identity/v3/projects/e95820335aef46a6a196c21938477174 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:45:01.762026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:01.762311 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:01.762588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:01.781261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:45:01.782466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:01.837920 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1352/2711] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:45:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:45:01.850397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:01.852169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:01.891822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:01.923030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db82954-ed74-44f9-8e63-cbdf5dd86990 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:01.928818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:01.929422 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:01.929706 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:01.929987 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:01.937985 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:01.939912 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7e95a-f9d0-4e8b-8d73-7bac7b2cefd7 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:01.948874 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1360/2712] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:45:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:45:11.830752 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=33,keystone:DELETE=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:11.945489 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=16 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:23.967153 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:23.968565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:24.107282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0268f-4bd8-47a7-af31-78c2c9fb22e8 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:24.112341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:24.112984 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:24.113313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:24.113588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:24.115353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:24.126792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:24.128429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:24.138204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1353/2713] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:45:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:45:34.134074 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=63 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:42.463336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edabab-4751-44c9-b6a5-0c25a3b4d4b6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:42.463851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edabab-4751-44c9-b6a5-0c25a3b4d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:42.463965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edabab-4751-44c9-b6a5-0c25a3b4d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:42.482965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-35edabab-4751-44c9-b6a5-0c25a3b4d4b6 None None] MFA Rules not processed for user `e02f332ba23a4a3cacb99f5c69a5b524`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:45:42.484162 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-35edabab-4751-44c9-b6a5-0c25a3b4d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:42.539249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1361/2714] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:45:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:45:42.855698 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:42.857167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:42.938549 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1220ea-f3c2-4606-81a7-703247f0e96d service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:42.942594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:42.943223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:42.943533 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:42.943974 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:42.948910 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:42.964851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:42.966295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06029692-0b11-4128-a8bd-b7ad6a730984 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:42.977539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1354/2715] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:45:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:45:52.532070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:52.972823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=35 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:46:10.443152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:46:10.443152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:46:10.443152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:46:10.462169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] MFA Rules not processed for user `81be4c83d3b04624ae11bcbf37a44309`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:46:10.463488 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:46:10.524093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1362/2716] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:46:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:46:20.515945 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:46:46.608929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fcf5-98b7-4c58-a5d2-db24af44f8a1 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:46:46.609565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fcf5-98b7-4c58-a5d2-db24af44f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:46:46.609565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859fcf5-98b7-4c58-a5d2-db24af44f8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:46:46.640927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0859fcf5-98b7-4c58-a5d2-db24af44f8a1 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:46:46.642422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0859fcf5-98b7-4c58-a5d2-db24af44f8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:46:46.657497 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1355/2717] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:46:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:46:46.704208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:46:46.716699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1e004-244c-4f09-adb7-869c91ceeb24 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:46:46.721689 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:46:46.722284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:46:46.722566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:46:46.722898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:46:46.731964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:46:46.733547 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0387a-3d79-4d1b-886d-a8f97c543a8c service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:46:46.741725 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1363/2718] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:46:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:46:56.651783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=10 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:46:56.738430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:46:59.838912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:46:59.839449 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:46:59.839639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:46:59.841867 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1356/2719] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:46:59.848493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516901-719b-4738-af75-82f6312a857c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:46:59.849515 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516901-719b-4738-af75-82f6312a857c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:46:59.849995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516901-719b-4738-af75-82f6312a857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:46:59.894708 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bc516901-719b-4738-af75-82f6312a857c None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:46:59.896084 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bc516901-719b-4738-af75-82f6312a857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:46:59.975408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1364/2720] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:46:59.986999 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:46:59.989275 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:00.047350 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddc26b-c7a9-40b5-8432-8526f23ddc85 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:00.050966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f978768101cd4e478b10b2f919383518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:00.052122 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.052395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.052660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.062516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:00.063969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5c5d-fc9f-4609-b6e0-f84d632984ad service glance] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:00.075483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1357/2721] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:46:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:00.144070 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.144346 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.144620 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.146697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1365/2722] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:47:00.151105 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532656a4-1bbd-4d23-873c-0813a7d9a58b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.151393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532656a4-1bbd-4d23-873c-0813a7d9a58b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.151667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532656a4-1bbd-4d23-873c-0813a7d9a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.171699 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-532656a4-1bbd-4d23-873c-0813a7d9a58b None None] MFA Rules not processed for user `8e2f7189b07f4024b7744322c707c2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:47:00.172909 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-532656a4-1bbd-4d23-873c-0813a7d9a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:00.179512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1358/2723] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:47:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:47:00.192141 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:00.194247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:00.208552 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b61fa-4f34-472a-8cba-f92198484efb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:00.212264 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:00.212951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.213298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.213637 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.221165 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:00.222597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0413542-73d8-42b1-8339-54ef85bb3d57 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:00.234285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1366/2724] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:00.327353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e06f-4bd8-4b10-a811-c41a0f60b2fc None None] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.327725 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e06f-4bd8-4b10-a811-c41a0f60b2fc None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.328071 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e06f-4bd8-4b10-a811-c41a0f60b2fc None None] PATH_INFO: `/` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.329731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1359/2725] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 15:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:47:00.335666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a529-434d-4d42-859f-06dc6c41301a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.336024 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a529-434d-4d42-859f-06dc6c41301a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.336374 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a529-434d-4d42-859f-06dc6c41301a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.359389 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0dc2a529-434d-4d42-859f-06dc6c41301a None None] MFA Rules not processed for user `8eb093eda2a14771aef59b6269ebe789`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:47:00.360642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0dc2a529-434d-4d42-859f-06dc6c41301a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:00.369934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1367/2726] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 15:47:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:47:00.380569 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e06f-4bd8-4b10-a811-c41a0f60b2fc None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:00.391891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e06f-4bd8-4b10-a811-c41a0f60b2fc None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:00.396182 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:00.396874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:00.397180 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:00.397453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:00.407561 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:00.408979 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:00.420243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1360/2727] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:02.155410 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:02.155976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:02.155976 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:02.181794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:47:02.183033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:02.190448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1368/2728] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 15:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:47:02.202977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:02.215362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97580298-94a2-4586-919a-744a6fb5c03e service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:02.220390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:02.220988 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:02.221267 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:02.221542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:02.230563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:02.232161 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081958a-b1dd-484a-9189-4704c33d8483 service swift] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:02.240954 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1361/2729] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 15:47:12.176621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=56 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:12.237478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=41 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:14.302696 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:14.313214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06089d4b-a2e9-4c2b-b4a6-0db5e9f82c70 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:14.319921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:14.320538 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:14.320819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:14.321092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:14.323520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:14.337642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:14.339125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:14.350532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1369/2730] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:15.815578 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:15.816092 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:15.816212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:15.836389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] MFA Rules not processed for user `fbfd83920ceb42598c2f2ba9b008a94a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:47:15.837601 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:15.844878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1362/2731] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 15:47:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:47:15.858189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:15.870457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71844334-355d-4139-8996-f7297d8d2691 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:15.875518 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e124d4d97e46b182279f9b24d2052d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:15.876105 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:15.876388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:15.876661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:15.884965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:15.886549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:15.894519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1370/2732] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 15:47:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 15:47:25.831461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:25.891291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=12 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:49.621178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:49.622563 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:49.698397 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930d93b-942f-41a7-aa2c-da66ba1fd233 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:49.702038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:49.702654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:49.702947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:49.703224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:49.704952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:49.716362 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:49.717939 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:49.727719 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1363/2733] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:50.069216 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:50.078584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef86688-d1a4-4880-9547-157d05dc21a2 service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:50.082927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:50.083559 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:50.083843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:50.084116 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:50.085844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:50.099238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:50.100666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:50.111838 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1371/2734] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:52.667696 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:52.669119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:52.687140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef066-cf78-490a-8663-474242e732dc service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:52.696229 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:52.696849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:52.697130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:52.697407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:52.699268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:52.712851 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:52.714471 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:52.728251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1364/2735] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:56.719962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:56.729621 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c063-24cd-4a62-a881-eb2ce82d9136 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:56.733245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:56.733837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:56.734115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:56.734388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:56.736157 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:56.818869 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:56.820405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:56.839775 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1372/2736] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:59.988935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:59.990353 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:00.005803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b69f3-6d5d-45e9-9a09-a42c421a7379 service nova] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.011019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.011659 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.011936 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.012208 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.013955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:00.027529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.029297 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.039023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.040563 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:00.041211 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1365/2737] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:00.054448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c25a4-ac2f-4036-93f2-6b0638e6ac9b service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.059169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.059822 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.060128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.060400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] PATH_INFO: `/v3/users/e02f332ba23a4a3cacb99f5c69a5b524` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.061520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e02f332ba23a4a3cacb99f5c69a5b524)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.063672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.110464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e02f332ba23a4a3cacb99f5c69a5b524'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.111035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e02f332ba23a4a3cacb99f5c69a5b524'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.118867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e02f332ba23a4a3cacb99f5c69a5b524'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.119493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e02f332ba23a4a3cacb99f5c69a5b524'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.126843 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3b6ccd99-ee0f-48c9-b66d-42b53bc8fff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e02f332ba23a4a3cacb99f5c69a5b524'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.144761 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1373/2738] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:00 2022] DELETE /identity/v3/users/e02f332ba23a4a3cacb99f5c69a5b524 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:00.323342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.335312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2347e-59de-4d8b-a3d5-ed4d2c02f881 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.392731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.393287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.393567 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.393844 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] PATH_INFO: `/v3/projects/9b9728b1ee454013a3968c27080cd98c` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.395715 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9728b1ee454013a3968c27080cd98c)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.397761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.443176 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9728b1ee454013a3968c27080cd98c', 'request_id': 'req-69eea49a-cd63-4f50-a6fa-946b201615f4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.448952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9728b1ee454013a3968c27080cd98c', 'request_id': 'req-69eea49a-cd63-4f50-a6fa-946b201615f4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.450032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:00.458341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9728b1ee454013a3968c27080cd98c', 'request_id': 'req-69eea49a-cd63-4f50-a6fa-946b201615f4'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.470109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] The token cache is being invalidate because project 9b9728b1ee454013a3968c27080cd98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:00.470393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.471033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-69eea49a-cd63-4f50-a6fa-946b201615f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:00.473133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1366/2739] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:00 2022] DELETE /identity/v3/projects/9b9728b1ee454013a3968c27080cd98c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:00.478376 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.478780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.479118 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.518561 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:00.530414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:00.531753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:00.590543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1374/2740] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:00.607081 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.607365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.607638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.626817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:00.628020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:00.636470 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1367/2741] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:00.640785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.652901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ac099-4f87-4ce5-ae60-c15fa1c348a8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.657442 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.658046 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.658328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.658598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.659290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:00.659595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.661717 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.673416 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1375/2742] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:48:00.677771 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.687087 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5aa5-5109-49a3-bef6-27cddaadfcdf None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.692383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.692977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.693255 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.693528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.694159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.696075 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.733029 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1368/2743] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:00 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:48:00.737109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.747114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31d5dc-24ad-4840-9688-b4b15ec06fd8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.803514 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.804112 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.804395 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.804672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.805398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.807370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.829846 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1376/2744] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:00 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:00.834285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.846259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46997-1b4d-43bd-9fa8-f473e19df5d8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.852312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.852935 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.853204 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.853476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.854132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.855763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.867065 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1369/2745] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:48:00.871103 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.882449 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd73628-22a6-4f74-83a0-df36ba07576d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.887167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.887800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.888081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.888350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.888994 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.890594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883c2d3-4e56-4d22-a78b-ea84890abc0d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.902318 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1377/2746] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:00.906691 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:00.917522 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981facc1-54a9-44c5-b8bb-c97b5086d06a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:00.922222 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:00.923033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.923380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.923726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] PATH_INFO: `/v3/projects/24bacae23a1b410dbf3e80d8a966ab4c/users/d5d52e557aad427c8ca6b1dcf3e38e3e/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.939855 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bacae23a1b410dbf3e80d8a966ab4c, user_id=d5d52e557aad427c8ca6b1dcf3e38e3e, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:00.942929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:00.956369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1370/2747] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:00 2022] PUT /identity/v3/projects/24bacae23a1b410dbf3e80d8a966ab4c/users/d5d52e557aad427c8ca6b1dcf3e38e3e/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:00.967306 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:00.967593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:00.967868 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:00.995988 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] MFA Rules not processed for user `d5d52e557aad427c8ca6b1dcf3e38e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:00.997212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:01.056758 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:01.063580 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1378/2748] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:01.067983 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.129109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffd93a-e53c-4acd-8651-749a7ec04217 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.133692 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.134284 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.134566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.134876 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.135496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.137395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.164368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1371/2749] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:01 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:01.168352 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.179778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca074-ee22-442a-8e1e-86b4db702210 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.235452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.236061 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.236347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.236628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.237255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.239214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.268379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1379/2750] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:01 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:01.273134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.283546 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710e8c-6a9e-40fc-92a1-97a0844491cb admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.288996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.289600 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.289880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.290154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.290861 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.292485 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.304545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1372/2751] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:01.308625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.319122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0c49a-df80-4afa-9938-4077b75012e0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.323807 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.324461 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.324741 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.325015 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] PATH_INFO: `/v3/projects/c19e5384932244918fff4527f62aea4b/users/d2d9de6d49b04c74acf021455e5c1ba6/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.340948 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19e5384932244918fff4527f62aea4b, user_id=d2d9de6d49b04c74acf021455e5c1ba6, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.343697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.357678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1380/2752] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:01 2022] PUT /identity/v3/projects/c19e5384932244918fff4527f62aea4b/users/d2d9de6d49b04c74acf021455e5c1ba6/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:01.362024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.372813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18034ee3-b8a4-48d5-82ef-72f615ace2cd admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.417196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:01.431069 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.431669 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.431950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.432225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.432868 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.434452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29832f4b-7e81-41a4-a773-b66b2b54cc9a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.444242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1373/2753] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:01.448424 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.458667 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12156-839d-4537-9202-b95a11619bca admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.463251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.463888 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.464161 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.464440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] PATH_INFO: `/v3/projects/c19e5384932244918fff4527f62aea4b/users/d2d9de6d49b04c74acf021455e5c1ba6/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.466304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19e5384932244918fff4527f62aea4b, user_id=d2d9de6d49b04c74acf021455e5c1ba6, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.468973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.479798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1381/2754] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:01 2022] PUT /identity/v3/projects/c19e5384932244918fff4527f62aea4b/users/d2d9de6d49b04c74acf021455e5c1ba6/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:01.491607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.491900 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.492178 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.519224 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] MFA Rules not processed for user `d2d9de6d49b04c74acf021455e5c1ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:01.520428 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:01.586446 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1374/2755] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:01.597003 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.599464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:01.655379 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb3970-1ff6-427b-bfef-19a49f901628 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.658953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.659548 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.660556 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.660828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.669555 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.670954 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b8930-d973-4ad0-9231-b74e992a8ad9 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.680135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1382/2756] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:01.817148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:01.877036 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d073a4-26d1-4dea-9379-31da98e01579 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:01.881665 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:01.882281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.882559 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.882877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] PATH_INFO: `/v3/users/d5d52e557aad427c8ca6b1dcf3e38e3e` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.883985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d52e557aad427c8ca6b1dcf3e38e3e)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:01.885916 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:01.931418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d52e557aad427c8ca6b1dcf3e38e3e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:01.932005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d52e557aad427c8ca6b1dcf3e38e3e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:01.932627 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d52e557aad427c8ca6b1dcf3e38e3e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:01.940816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d52e557aad427c8ca6b1dcf3e38e3e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:01.953325 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d52e557aad427c8ca6b1dcf3e38e3e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:01.954736 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:01.969614 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1375/2757] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:01 2022] DELETE /identity/v3/users/d5d52e557aad427c8ca6b1dcf3e38e3e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:01.974828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:01.975124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:01.975402 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:01.994268 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:01.995498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:02.042780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:02.056328 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1383/2758] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:02.066856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:02.068301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:02.124553 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c07dc-79e7-46a6-a770-2c53a9d73c69 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:02.128190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:02.129379 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:02.129661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:02.129932 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:02.138877 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:02.140305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:02.150358 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1376/2759] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:02.421852 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:02.423333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:02.437783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd2a76-4e2b-4fa7-bf80-6c39d05c5967 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:02.443891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:02.444652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:02.444802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:02.445080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] PATH_INFO: `/v3/projects/24bacae23a1b410dbf3e80d8a966ab4c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:02.446265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bacae23a1b410dbf3e80d8a966ab4c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:02.448212 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:02.496375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bacae23a1b410dbf3e80d8a966ab4c', 'request_id': 'req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.501809 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bacae23a1b410dbf3e80d8a966ab4c', 'request_id': 'req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.508549 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bacae23a1b410dbf3e80d8a966ab4c', 'request_id': 'req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.521012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] The token cache is being invalidate because project 24bacae23a1b410dbf3e80d8a966ab4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:02.521304 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.521929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.523465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1384/2760] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:02 2022] DELETE /identity/v3/projects/24bacae23a1b410dbf3e80d8a966ab4c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:02.527551 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:02.537364 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a1882-ef96-4bf0-b1cf-c12a8b4aca5d service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:02.592550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:02.593190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:02.593463 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:02.593731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] PATH_INFO: `/v3/users/d2d9de6d49b04c74acf021455e5c1ba6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:02.594860 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d9de6d49b04c74acf021455e5c1ba6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:02.596756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:02.642372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d9de6d49b04c74acf021455e5c1ba6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.643032 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d9de6d49b04c74acf021455e5c1ba6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.643664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d9de6d49b04c74acf021455e5c1ba6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.651226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d9de6d49b04c74acf021455e5c1ba6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.663321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-f0bd5013-c9dc-457c-aab5-0c73f2562165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d9de6d49b04c74acf021455e5c1ba6'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:02.681880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1377/2761] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:02 2022] DELETE /identity/v3/users/d2d9de6d49b04c74acf021455e5c1ba6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:02.950694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:02.952370 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:02.967891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2131-fb28-4c78-b4d6-33bfdcc65db4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.025499 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.026128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.026411 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.026728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] PATH_INFO: `/v3/projects/c19e5384932244918fff4527f62aea4b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.028007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19e5384932244918fff4527f62aea4b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.029936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.083927 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19e5384932244918fff4527f62aea4b', 'request_id': 'req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:03.089403 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19e5384932244918fff4527f62aea4b', 'request_id': 'req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:03.095474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19e5384932244918fff4527f62aea4b', 'request_id': 'req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:03.109195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] The token cache is being invalidate because project c19e5384932244918fff4527f62aea4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:03.109480 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:03.110092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-72a9243e-5c8a-40ae-8209-0c0c5b9dae08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:03.111633 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1385/2762] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:02 2022] DELETE /identity/v3/projects/c19e5384932244918fff4527f62aea4b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:48:03.117272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.117774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.117873 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.137050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:03.138241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:03.143458 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:03.199452 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1378/2763] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:48:03.208037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.208336 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.208677 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.225567 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:03.232148 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:03.233953 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:03.242128 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1386/2764] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:03.246427 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.256206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042dfbb-08d8-447e-9a3f-0a18a76bff6c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.260991 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.261591 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.261874 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.262151 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.262840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:03.263148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.264734 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.276831 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1379/2765] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:48:03.281177 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.291196 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca834-c1fd-4418-8a24-7053c52c6c29 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.295803 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.296396 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.296678 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.296952 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.297788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.299745 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.327132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1387/2766] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:03.331289 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.341783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986796-3182-479d-b76b-598d20c78282 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.403022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.403022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.403022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.403022 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.403527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.403527 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.424816 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1380/2767] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:48:03.428941 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.439397 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42634f6-f01d-4302-9823-e1ee753062ee admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.444251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.444850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.445133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.445408 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.446056 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.447701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.458627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1388/2768] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:03.462481 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.473114 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d9976-0b5c-4378-9a69-da01474fe8a2 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.478054 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.478667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.478964 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.479239 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.479901 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.481492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be330aef-eefb-489b-8bc8-faa9276935f8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.493226 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1381/2769] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:03.497348 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.510113 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d5b58-f317-4488-9b33-f6ed26970feb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.515011 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.515672 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.515956 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.516237 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] PATH_INFO: `/v3/projects/728186f18b1b4b638285906fdff67862/users/0d8deb0df08b4eff842bd1b6f1c79684/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.531498 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728186f18b1b4b638285906fdff67862, user_id=0d8deb0df08b4eff842bd1b6f1c79684, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.534093 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.547199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1389/2770] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:03 2022] PUT /identity/v3/projects/728186f18b1b4b638285906fdff67862/users/0d8deb0df08b4eff842bd1b6f1c79684/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:03.558080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.558369 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.558668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.586487 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] MFA Rules not processed for user `0d8deb0df08b4eff842bd1b6f1c79684`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:03.587726 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:03.650564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1382/2771] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:03.655074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.716270 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45e39e-8aef-478a-b912-1845dcbef2e0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.721049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.721649 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.721936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.722214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.722876 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.724798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.751281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1390/2772] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:03.756130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.766973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc4ada-a590-4403-a4ad-62bddbba1f2c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.771134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:03.825801 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.826332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.826761 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.827038 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.827666 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.829556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.853950 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1383/2773] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:03 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:48:03.858121 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.869703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d962b00-4aec-48e0-9628-bb081ebdc90e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.874591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.875221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.875506 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.875780 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.876459 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.878074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.893783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1391/2774] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:03.895565 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.906982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ad35c-2c0f-4f04-ae2c-c4b68c822098 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:03.912550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:03.913240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:03.913529 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:03.913803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] PATH_INFO: `/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6/users/35a0197b5ea744729376058f1d300a47/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:03.929899 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e98b61dbe864b6bb63b498f4e01cec6, user_id=35a0197b5ea744729376058f1d300a47, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:03.932705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:03.947047 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1384/2775] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:03 2022] PUT /identity/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6/users/35a0197b5ea744729376058f1d300a47/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:03.951249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:03.962591 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35f65a-676d-4e6f-b7e8-ca2d5ec2784d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:04.020208 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:04.020802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:04.021080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:04.021377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:04.022021 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:04.023635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeea280-1304-4ec9-bf76-bb3efe887999 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:04.033935 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1392/2776] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:04.038948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:04.049713 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba03b-7b20-43ba-86f6-82d8ea913532 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:04.054490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:04.055171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:04.055451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:04.055724 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] PATH_INFO: `/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6/users/35a0197b5ea744729376058f1d300a47/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:04.057737 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e98b61dbe864b6bb63b498f4e01cec6, user_id=35a0197b5ea744729376058f1d300a47, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:04.060396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:04.071792 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1385/2777] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:04 2022] PUT /identity/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6/users/35a0197b5ea744729376058f1d300a47/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:48:04.084030 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:04.084249 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:04.084525 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:04.109810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:04.114146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] MFA Rules not processed for user `35a0197b5ea744729376058f1d300a47`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:04.115964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:04.183595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1393/2778] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:04.195782 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:04.197542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:04.255346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad98d56-9e41-4313-a7bc-a2075ffd391a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:04.258906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:04.259498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:04.259774 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:04.260044 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:04.268694 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:04.270408 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:04.279632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1386/2779] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:04.355500 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:04.366145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffd663-7980-44e7-8b38-c67b9c6a9902 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:04.369871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:04.370501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:04.370825 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:04.371114 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:04.427892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:04.429335 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:04.440519 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1394/2780] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:06.339623 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:06.341256 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:06.416667 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1d84-770e-48f4-9df3-54ad89158ea7 service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:06.420525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:06.421134 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:06.421461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:06.421679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:06.423622 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:06.435346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:06.436783 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fe6a1-92c6-4ead-bcf6-b17f5443cbde service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:06.454973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1387/2781] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:12.472631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:12.482081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903fef3-b8e2-4521-8a89-2f545edd320d service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:12.485902 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:12.486521 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:12.486851 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:12.487130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:12.488934 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:12.548619 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:12.550009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:12.558991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1395/2782] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:16.443757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:17.246213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:17.246499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:17.246802 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:17.267377 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] MFA Rules not processed for user `81be4c83d3b04624ae11bcbf37a44309`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:17.268580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:17.277368 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1388/2783] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:17.596962 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:17.598429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:17.610337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d61ca-e6f7-4485-9612-82c3e88eec62 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:17.613938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:17.614535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:17.614841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:17.615117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:17.617298 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:17.626457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:17.627916 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:17.636553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1396/2784] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:18.133508 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:18.135109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:18.240006 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990905a2-a327-4c1d-95e6-7ff0a115bdf6 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:18.247195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:18.247825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:18.248109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:18.248389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:18.250140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:18.262307 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:18.263926 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:18.274764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1389/2785] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:22.321671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:22.334102 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5cacb-39a1-48a8-8d31-dda3545a4944 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:22.339674 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:22.340272 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:22.340551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:22.340823 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:22.342693 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:22.357012 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:22.358791 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:22.371440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1397/2786] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:24.207412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:24.218564 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42331-5231-4cd0-a366-f7153f735ac7 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:24.223662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:24.224606 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:24.225670 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:24.226206 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:24.228486 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:24.300773 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:24.302295 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:24.312865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1390/2787] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:48:25.897504 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:25.899188 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:25.924638 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8eff7-ba30-4ba9-826a-9bd0124cdf5a service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:25.932255 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:25.932901 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:25.933343 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:25.933624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] PATH_INFO: `/v3/users/90be4ab702e14f4387b0883dead72d9f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:25.935595 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90be4ab702e14f4387b0883dead72d9f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:25.938611 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:25.941681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:25.996331 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90be4ab702e14f4387b0883dead72d9f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:25.997134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90be4ab702e14f4387b0883dead72d9f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.007254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90be4ab702e14f4387b0883dead72d9f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.008023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90be4ab702e14f4387b0883dead72d9f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.015810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90be4ab702e14f4387b0883dead72d9f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.037421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1398/2788] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:25 2022] DELETE /identity/v3/users/90be4ab702e14f4387b0883dead72d9f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:26.211476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:26.221263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1926-3d21-405f-93cc-95f21cbb2f2a service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:26.280799 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:26.281424 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:26.281703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:26.281977 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] PATH_INFO: `/v3/projects/3760b78ba8584e6c81c46d48113fcd45` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:26.283128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3760b78ba8584e6c81c46d48113fcd45)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:26.285025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:26.307116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:26.335309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3760b78ba8584e6c81c46d48113fcd45', 'request_id': 'req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.340823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3760b78ba8584e6c81c46d48113fcd45', 'request_id': 'req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.347157 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3760b78ba8584e6c81c46d48113fcd45', 'request_id': 'req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.359881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] The token cache is being invalidate because project 3760b78ba8584e6c81c46d48113fcd45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:26.360166 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.360828 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.362346 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1391/2789] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:26 2022] DELETE /identity/v3/projects/3760b78ba8584e6c81c46d48113fcd45 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:26.367203 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:26.368629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:26.382576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebede36-ee77-48fa-a7f4-7f7e90ac5dcd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:26.439748 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:26.440296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:26.440576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:26.440850 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] PATH_INFO: `/v3/users/41b4bacdcd76476d8ef084eeb58f2569` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:26.441990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b4bacdcd76476d8ef084eeb58f2569)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:26.444737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:26.498532 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b4bacdcd76476d8ef084eeb58f2569'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.499421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b4bacdcd76476d8ef084eeb58f2569'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.508275 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b4bacdcd76476d8ef084eeb58f2569'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.508997 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b4bacdcd76476d8ef084eeb58f2569'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.516732 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b4bacdcd76476d8ef084eeb58f2569'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.530418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:26.535133 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1399/2790] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:26 2022] DELETE /identity/v3/users/41b4bacdcd76476d8ef084eeb58f2569 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:48:26.712271 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:26.713727 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:26.726576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef3a2-3a3c-483f-b494-788a6dd8c19a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:26.783215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:26.783847 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:26.784128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:26.784403 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] PATH_INFO: `/v3/projects/18adf863101244fe954a347593663444` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:26.785512 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18adf863101244fe954a347593663444)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:26.787509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:26.833046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18adf863101244fe954a347593663444', 'request_id': 'req-0c3f20a0-9f65-478b-85fb-d1019ab01980'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.839728 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18adf863101244fe954a347593663444', 'request_id': 'req-0c3f20a0-9f65-478b-85fb-d1019ab01980'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.846431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18adf863101244fe954a347593663444', 'request_id': 'req-0c3f20a0-9f65-478b-85fb-d1019ab01980'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.860146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] The token cache is being invalidate because project 18adf863101244fe954a347593663444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:26.860554 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.861195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:26.862744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1392/2791] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:26 2022] DELETE /identity/v3/projects/18adf863101244fe954a347593663444 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:36.263426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:36.265051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:36.344206 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a1a3-6a54-4d40-acfb-f62ba8a8ec6a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:36.406928 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:36.407566 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:36.407847 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:36.408122 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:36.409875 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:36.424628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:36.429058 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:36.438278 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1400/2792] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:36.856912 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:40.165588 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:40.167155 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:40.183925 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f20a0-9f65-478b-85fb-d1019ab01980 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:40.189337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eb093eda2a14771aef59b6269ebe789', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:40.189938 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:40.190216 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:40.190488 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:40.192240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:40.204833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:40.206418 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce87bc1-1c15-48e3-b952-b1251b3e7c5f service nova] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:40.217537 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1393/2793] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:46.433723 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=62 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:49.607577 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:49.608983 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:49.673979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718ee2-97d1-4ef9-bccc-387bf1ea3a27 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:49.679043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:49.679668 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:49.679950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:49.680226 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] PATH_INFO: `/v3/users/0d8deb0df08b4eff842bd1b6f1c79684` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:49.681345 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8deb0df08b4eff842bd1b6f1c79684)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:49.683324 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:49.730202 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8deb0df08b4eff842bd1b6f1c79684'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:49.730890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8deb0df08b4eff842bd1b6f1c79684'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:49.737636 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8deb0df08b4eff842bd1b6f1c79684'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:49.738265 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8deb0df08b4eff842bd1b6f1c79684'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:49.746064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8deb0df08b4eff842bd1b6f1c79684'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:49.764385 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1401/2794] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:49 2022] DELETE /identity/v3/users/0d8deb0df08b4eff842bd1b6f1c79684 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:49.768968 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:49.769483 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:49.769598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:49.788327 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:49.789531 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:49.847482 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1394/2795] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:49.857584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:49.859071 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:49.916057 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef198-abaa-4c6d-89a1-33388fbe974f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:49.922698 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:49.923314 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:49.923592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:49.923864 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:49.935681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:49.937059 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:49.947301 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1402/2796] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:50.211005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:50.212892 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:50.225391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbbb21-fa77-4835-8120-c66ae47b0eca None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:50.230277 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:50.231078 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:50.231359 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:50.231628 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] PATH_INFO: `/v3/projects/728186f18b1b4b638285906fdff67862` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:50.232888 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728186f18b1b4b638285906fdff67862)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:50.234903 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:50.282746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728186f18b1b4b638285906fdff67862', 'request_id': 'req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.288333 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728186f18b1b4b638285906fdff67862', 'request_id': 'req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.294341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728186f18b1b4b638285906fdff67862', 'request_id': 'req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.307763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] The token cache is being invalidate because project 728186f18b1b4b638285906fdff67862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:50.308034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.308648 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.310120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1395/2797] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:50 2022] DELETE /identity/v3/projects/728186f18b1b4b638285906fdff67862 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:50.314474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:50.324959 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599c179-b41e-4061-bb77-833ad9fede1d service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:50.342942 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:50.383078 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:50.383701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:50.383977 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:50.384241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] PATH_INFO: `/v3/users/35a0197b5ea744729376058f1d300a47` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:50.385421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a0197b5ea744729376058f1d300a47)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:50.387472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:50.434414 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a0197b5ea744729376058f1d300a47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.435134 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a0197b5ea744729376058f1d300a47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.441999 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a0197b5ea744729376058f1d300a47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.442658 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a0197b5ea744729376058f1d300a47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.450466 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9011c5c1-6074-42a6-8536-a1eff0045999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a0197b5ea744729376058f1d300a47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.468077 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1403/2798] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:50 2022] DELETE /identity/v3/users/35a0197b5ea744729376058f1d300a47 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:50.739604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:50.741164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:50.758599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d1bf-da77-4d0e-8a50-9913e805ece0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:50.816186 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:50.816813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:50.817091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:50.817365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] PATH_INFO: `/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:50.819019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e98b61dbe864b6bb63b498f4e01cec6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:50.820919 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:50.832188 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:50.868169 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e98b61dbe864b6bb63b498f4e01cec6', 'request_id': 'req-d77ac035-4e50-4895-9a1f-8df49d7d83c3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.873749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e98b61dbe864b6bb63b498f4e01cec6', 'request_id': 'req-d77ac035-4e50-4895-9a1f-8df49d7d83c3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.879864 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e98b61dbe864b6bb63b498f4e01cec6', 'request_id': 'req-d77ac035-4e50-4895-9a1f-8df49d7d83c3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.893594 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] The token cache is being invalidate because project 6e98b61dbe864b6bb63b498f4e01cec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:50.893880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.894538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-d77ac035-4e50-4895-9a1f-8df49d7d83c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:50.896050 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1396/2799] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:50 2022] DELETE /identity/v3/projects/6e98b61dbe864b6bb63b498f4e01cec6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:50.901712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:50.902178 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:50.902308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:50.922080 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:50.923300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:50.984686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1404/2800] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:51.046959 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.047251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.047525 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.067434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:51.070049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:51.077940 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1397/2801] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:51.082107 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.091841 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6c2bb-ab63-4f57-b77c-4875a8233ebc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.096391 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.096974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.097276 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.097543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.098182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:51.098478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.100130 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.111516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1405/2802] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:48:51.116328 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.133390 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c92f-3962-4470-9a82-b1756cfa735b None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.138056 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.138662 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.138955 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.139227 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.139854 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.141735 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.168429 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1398/2803] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:51.172936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.183526 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bdd765-e301-4c4b-822d-b66153305120 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.209302 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:51.242608 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.243150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.243430 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.243703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.244337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.246284 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.267974 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1406/2804] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:51.272764 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.282957 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefa0ec-f649-4ed3-943e-d4a23665f192 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.287714 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.288301 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.288580 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.288850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.289490 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.291108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.300848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1399/2805] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:51.305458 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.315817 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf5aa-0746-4e7c-b8ec-2868851b1888 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.321445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.322043 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.322325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.322602 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.323285 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.325074 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb762d-a828-48df-a429-2db5910e69ce admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.335863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1407/2806] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:51.340073 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.349890 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831cfa-4b85-4010-82cd-62793ef6f004 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.354905 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.355542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.355821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.356095 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] PATH_INFO: `/v3/projects/1cc505d0cad84da398c947ef65493e73/users/6578d9fc507b4df28c3844c1ca0f673b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.370757 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc505d0cad84da398c947ef65493e73, user_id=6578d9fc507b4df28c3844c1ca0f673b, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.373404 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.386555 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1400/2807] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:51 2022] PUT /identity/v3/projects/1cc505d0cad84da398c947ef65493e73/users/6578d9fc507b4df28c3844c1ca0f673b/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:51.396405 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.396718 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.397033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.424692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] MFA Rules not processed for user `6578d9fc507b4df28c3844c1ca0f673b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:51.425960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:51.488768 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1408/2808] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:51.494187 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.558535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678366f-b518-412d-b38f-b632f6331d0e admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.563196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.563823 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.564108 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.564385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.565017 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.567033 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.593077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1401/2809] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:51.597238 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.608749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7974e-788b-4f18-8552-24f96e1df08e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.663142 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.663737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.664020 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.664294 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.664914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.666858 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.688369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1409/2810] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:51.692478 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.702881 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ff66e-1818-4cab-aaeb-5fc9fd927fb7 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.707516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.708109 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.708391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.708664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.709320 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.711119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.726021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1402/2811] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:51.730975 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.741984 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ff55d-eb27-49fa-9873-17ca53a8a0fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.746691 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.747303 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.747589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.747870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] PATH_INFO: `/v3/projects/e816bf7086844abdbd6db92b312ef550/users/5d364e6456ac4da2a5abe4f3111a039a/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.763205 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e816bf7086844abdbd6db92b312ef550, user_id=5d364e6456ac4da2a5abe4f3111a039a, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.765881 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.780200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1410/2812] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:51 2022] PUT /identity/v3/projects/e816bf7086844abdbd6db92b312ef550/users/5d364e6456ac4da2a5abe4f3111a039a/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:51.784099 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.794098 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c461d9-4e14-48d6-9ee6-461da731ef85 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.801953 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:51.852576 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.853542 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.853700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.853971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.854605 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.856281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69478b-16da-46b3-9386-2e76eb02b97f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.866279 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1403/2813] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:51.870616 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:51.876049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:51.882792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23989665-6dbd-46f0-ab27-7da9e00e5742 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:51.888598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:51.890546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.890546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.890546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] PATH_INFO: `/v3/projects/e816bf7086844abdbd6db92b312ef550/users/5d364e6456ac4da2a5abe4f3111a039a/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.892081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e816bf7086844abdbd6db92b312ef550, user_id=5d364e6456ac4da2a5abe4f3111a039a, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:51.896509 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:51.908964 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1411/2814] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:51 2022] PUT /identity/v3/projects/e816bf7086844abdbd6db92b312ef550/users/5d364e6456ac4da2a5abe4f3111a039a/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:48:51.919283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:51.919574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:51.919850 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:51.947946 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] MFA Rules not processed for user `5d364e6456ac4da2a5abe4f3111a039a`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:51.949148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:52.016143 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1404/2815] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:52.028947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.030694 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:52.088737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aad725-246a-4246-80d5-9a8eb597442c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:52.092377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:52.092973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:52.093254 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:52.093528 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:52.102369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:52.104096 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:52.113378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1412/2816] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:52.420702 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.430136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ab3d6-3149-494f-9d8a-385722dc0051 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:52.434661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d364e6456ac4da2a5abe4f3111a039a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e816bf7086844abdbd6db92b312ef550', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:52.435262 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:52.435543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:52.435817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:52.436461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:52.436762 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:52.438337 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:52.450331 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1405/2817] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:48:52.454450 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.464827 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ecc34-7f91-4803-9e52-84c695f8d655 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:52.469333 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d364e6456ac4da2a5abe4f3111a039a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e816bf7086844abdbd6db92b312ef550', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:52.469944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:52.470221 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:52.470493 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:52.471167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:52.473052 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:52.498967 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1413/2818] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:52 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:52.502990 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.513967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce9143-7392-4989-a8ad-5607a2576561 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:52.571338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d364e6456ac4da2a5abe4f3111a039a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e816bf7086844abdbd6db92b312ef550', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:52.571949 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:52.572234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:52.572514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:52.573164 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:52.573473 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:52.575154 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:52.587528 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1406/2819] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:48:52.625050 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.635436 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e090d16-9182-4726-bc92-b1e3509a9ef0 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:52.640064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d364e6456ac4da2a5abe4f3111a039a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e816bf7086844abdbd6db92b312ef550', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:52.640683 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:52.640968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:52.641245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] PATH_INFO: `/v3/projects/311d53d7a8a349c4802b288e1e5893b7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:52.642384 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorizing `identity:delete_project(project_id=311d53d7a8a349c4802b288e1e5893b7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:52.644364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:52.690879 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311d53d7a8a349c4802b288e1e5893b7', 'request_id': 'req-54457cdc-f5e8-4986-bc26-411a9382214e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:52.696441 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311d53d7a8a349c4802b288e1e5893b7', 'request_id': 'req-54457cdc-f5e8-4986-bc26-411a9382214e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:52.703064 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311d53d7a8a349c4802b288e1e5893b7', 'request_id': 'req-54457cdc-f5e8-4986-bc26-411a9382214e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:52.715712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] The token cache is being invalidate because project 311d53d7a8a349c4802b288e1e5893b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:52.715990 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:52.716629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-54457cdc-f5e8-4986-bc26-411a9382214e tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:52.718132 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1414/2820] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:52 2022] DELETE /identity/v3/projects/311d53d7a8a349c4802b288e1e5893b7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:52.914120 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:52.915933 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:52.930184 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=100 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:52.980785 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb479-494e-4198-8a14-71c1f6c3b577 tempest-VolumeQuotaClassesTest-1924518598 tempest-VolumeQuotaClassesTest-1924518598-project-admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:53.035128 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:53.035754 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:53.036034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:53.036312 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] PATH_INFO: `/v3/users/6578d9fc507b4df28c3844c1ca0f673b` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:53.037453 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] RBAC: Authorizing `identity:delete_user(user_id=6578d9fc507b4df28c3844c1ca0f673b)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:53.039457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:53.085148 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6578d9fc507b4df28c3844c1ca0f673b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.085786 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6578d9fc507b4df28c3844c1ca0f673b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.086451 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6578d9fc507b4df28c3844c1ca0f673b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.093657 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6578d9fc507b4df28c3844c1ca0f673b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.105907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6578d9fc507b4df28c3844c1ca0f673b'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.118192 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1407/2821] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:52 2022] DELETE /identity/v3/users/6578d9fc507b4df28c3844c1ca0f673b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:53.122890 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:53.123377 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:53.123491 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:53.142143 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:53.143364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:53.201625 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:53.222754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1415/2822] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:53.232917 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:53.234342 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:53.277947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:53.292489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b1252-fcde-4dca-bd59-81f99d2e68da admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:53.296140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:53.296738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:53.297020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:53.297296 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:53.306630 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:53.308067 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:53.318489 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1408/2823] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:53.599535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:53.601066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:53.613712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4cecf-1b6e-499a-95b7-4fff3b3e0abf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:53.619426 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:53.620066 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:53.620350 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:53.620627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] PATH_INFO: `/v3/projects/1cc505d0cad84da398c947ef65493e73` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:53.621787 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc505d0cad84da398c947ef65493e73)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:53.623764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:53.671063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc505d0cad84da398c947ef65493e73', 'request_id': 'req-388061c1-b772-4f74-9ff2-eecde87bccf3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.676353 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc505d0cad84da398c947ef65493e73', 'request_id': 'req-388061c1-b772-4f74-9ff2-eecde87bccf3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.683026 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc505d0cad84da398c947ef65493e73', 'request_id': 'req-388061c1-b772-4f74-9ff2-eecde87bccf3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.695152 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] The token cache is being invalidate because project 1cc505d0cad84da398c947ef65493e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:53.695434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.696049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.697517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1416/2824] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:53 2022] DELETE /identity/v3/projects/1cc505d0cad84da398c947ef65493e73 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:53.701581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:53.711367 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa7593-fe28-43cd-9602-6b3382694a75 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:53.767205 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:53.767833 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:53.768116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:53.768396 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] PATH_INFO: `/v3/users/5d364e6456ac4da2a5abe4f3111a039a` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:53.769515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d364e6456ac4da2a5abe4f3111a039a)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:53.771494 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:53.816996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d364e6456ac4da2a5abe4f3111a039a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.817634 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d364e6456ac4da2a5abe4f3111a039a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.818225 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d364e6456ac4da2a5abe4f3111a039a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.825660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d364e6456ac4da2a5abe4f3111a039a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.837631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3caaf97d-dd61-478a-9103-c76525582a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d364e6456ac4da2a5abe4f3111a039a'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:53.850462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1409/2825] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:53 2022] DELETE /identity/v3/users/5d364e6456ac4da2a5abe4f3111a039a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:54.126135 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.127590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:54.141092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388061c1-b772-4f74-9ff2-eecde87bccf3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.196986 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.197622 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.197903 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.198181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] PATH_INFO: `/v3/projects/e816bf7086844abdbd6db92b312ef550` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.199339 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e816bf7086844abdbd6db92b312ef550)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.201381 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.247482 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e816bf7086844abdbd6db92b312ef550', 'request_id': 'req-d4d35e33-2073-486c-b53c-74407211b248'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:54.252828 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e816bf7086844abdbd6db92b312ef550', 'request_id': 'req-d4d35e33-2073-486c-b53c-74407211b248'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:54.258783 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e816bf7086844abdbd6db92b312ef550', 'request_id': 'req-d4d35e33-2073-486c-b53c-74407211b248'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:54.270800 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] The token cache is being invalidate because project e816bf7086844abdbd6db92b312ef550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:48:54.271085 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:54.272378 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d4d35e33-2073-486c-b53c-74407211b248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:54.274151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1417/2826] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:48:54 2022] DELETE /identity/v3/projects/e816bf7086844abdbd6db92b312ef550 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:54.279961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.280431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.280570 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.301215 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:54.302425 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:54.364281 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1410/2827] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:54.373463 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.373692 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.373969 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.388295 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:54.394269 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:54.396404 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:54.405440 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1418/2828] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:48:54.409824 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.415378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:54.421664 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31870dd8-ad7c-4f76-a072-33049597a133 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.427037 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.427660 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.427961 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.428231 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.428865 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:48:54.429165 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.430766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.442413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1411/2829] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:48:54.447421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.456587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21db22-d7ae-42ce-9645-931b00899a6b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.461950 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.462539 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.462855 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.463126 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.463753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.465643 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.491766 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1419/2830] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:54.496043 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.506675 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d730f1-8321-4464-9ebf-dc88280141c1 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.562398 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.563025 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.563305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.563577 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.564202 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.566091 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.588045 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1412/2831] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:54.592160 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.602289 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa8545-f6ab-4963-a134-9f2bcd2cb118 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.607139 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.608697 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.608973 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.609242 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.609886 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.611501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.621516 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1420/2832] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:54.625524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.636110 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec7b1-2c94-4cd2-8162-5d5db6c7a691 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.640677 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.641264 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.641547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.641827 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.642466 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.644096 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66390-cac1-4d56-85a2-8533564da946 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.655210 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1413/2833] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:54.659181 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.669247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d10e6-0a56-4ca2-9b63-1b6364d0e48a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.673965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.674605 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.674963 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.675241 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] PATH_INFO: `/v3/projects/c4d2fd910dd845d5bbc71958f8100581/users/12157843aa25454b88b0eb64311741e3/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.691371 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d2fd910dd845d5bbc71958f8100581, user_id=12157843aa25454b88b0eb64311741e3, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.694788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.707231 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1421/2834] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:54 2022] PUT /identity/v3/projects/c4d2fd910dd845d5bbc71958f8100581/users/12157843aa25454b88b0eb64311741e3/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:54.718145 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.718461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.718769 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.745744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] MFA Rules not processed for user `12157843aa25454b88b0eb64311741e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:54.746996 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:54.809749 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1414/2835] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:54.814207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.879125 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4b0e3-9350-42ee-901a-842a09c3e5fb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.884300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.884900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.885182 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.885457 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.886081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.888040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:54.914320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1422/2836] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:48:54.918382 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:54.929575 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6854ef-a95b-468f-873b-86b2e6a2277c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:54.979360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:54.986562 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:54.987194 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:54.987474 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:54.987746 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:54.988360 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:54.990882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.012254 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1415/2837] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:48:54 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:48:55.016429 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.026752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c40b0-99e1-417c-874f-40d43421512d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.032468 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.033029 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.033308 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.033632 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.034195 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.035944 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.047351 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1423/2838] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:55.051509 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.061834 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27007-7fd1-4cb8-9af7-b0df862d201b admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.066371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.067052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.067329 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.067604 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] PATH_INFO: `/v3/projects/ab40a6072b194ff0916249f8b70c9177/users/1a8e9aa35a164c1191613e8520529499/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.082430 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab40a6072b194ff0916249f8b70c9177, user_id=1a8e9aa35a164c1191613e8520529499, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.085130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.097947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1416/2839] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:55 2022] PUT /identity/v3/projects/ab40a6072b194ff0916249f8b70c9177/users/1a8e9aa35a164c1191613e8520529499/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:48:55.102146 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.112584 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8f048-dab6-409f-8b37-37017e462f92 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.169751 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.170383 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.170550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.170854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.171495 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.173082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa9da-3fa6-4c77-a1f3-3523ee419e6f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.185946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1424/2840] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:48:55.192413 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.203515 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baabae-bef7-463f-ba0d-a84cb47991bf admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.208175 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.208825 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.209285 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.209572 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] PATH_INFO: `/v3/projects/ab40a6072b194ff0916249f8b70c9177/users/1a8e9aa35a164c1191613e8520529499/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.211336 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab40a6072b194ff0916249f8b70c9177, user_id=1a8e9aa35a164c1191613e8520529499, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.213971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.227119 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1417/2841] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:48:55 2022] PUT /identity/v3/projects/ab40a6072b194ff0916249f8b70c9177/users/1a8e9aa35a164c1191613e8520529499/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:55.238572 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.238894 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.239199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.264434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:55.267729 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] MFA Rules not processed for user `1a8e9aa35a164c1191613e8520529499`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:55.269485 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:55.338296 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1425/2842] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:55.349019 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.350895 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:55.408132 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51942a9-e2c7-4ff4-8038-84ce023eacb6 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.411777 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.412476 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.412760 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.413024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.421853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.423252 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.432545 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1418/2843] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:55.510189 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:55.520329 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd3644-54da-49d7-82e2-e568e11f44dc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:55.524792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:55.525372 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:55.525652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:55.525924 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:55.581863 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:55.583341 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f73b54-05b4-4751-856e-75817154d667 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:55.592520 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1426/2844] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:59.631832 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:59.692061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97ab91-74c7-4f7f-9c93-9c0237ac76eb service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:59.697344 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:59.697960 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:59.698242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:59.698516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] PATH_INFO: `/v3/users/12157843aa25454b88b0eb64311741e3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:59.699647 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12157843aa25454b88b0eb64311741e3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:59.701539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:59.746290 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12157843aa25454b88b0eb64311741e3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:59.746971 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12157843aa25454b88b0eb64311741e3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:59.747582 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12157843aa25454b88b0eb64311741e3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:59.754479 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12157843aa25454b88b0eb64311741e3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:59.755701 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:59.769381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12157843aa25454b88b0eb64311741e3'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:48:59.782052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1419/2845] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:48:59 2022] DELETE /identity/v3/users/12157843aa25454b88b0eb64311741e3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:48:59.786535 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:59.787051 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:59.787239 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:59.805545 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:59.806753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:59.865728 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1427/2846] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:59.875317 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:59.876693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:59.934492 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b18aa-d6b6-4fd1-8129-33eaff35d943 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:59.938199 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:59.938843 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:59.939125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:59.939401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:59.949024 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:59.950448 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:59.960326 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1420/2847] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:00.227749 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:00.229452 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:00.244938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8150a-cb3c-4c41-b64a-3cfd80a0d3c5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:00.249530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:00.250227 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:00.250435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:00.250739 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] PATH_INFO: `/v3/projects/c4d2fd910dd845d5bbc71958f8100581` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:00.251867 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d2fd910dd845d5bbc71958f8100581)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:00.253814 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:00.288764 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:00.302004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d2fd910dd845d5bbc71958f8100581', 'request_id': 'req-44e1a49e-c76c-4048-910a-0a9b7914701d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.308342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d2fd910dd845d5bbc71958f8100581', 'request_id': 'req-44e1a49e-c76c-4048-910a-0a9b7914701d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.314662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d2fd910dd845d5bbc71958f8100581', 'request_id': 'req-44e1a49e-c76c-4048-910a-0a9b7914701d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.328537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] The token cache is being invalidate because project c4d2fd910dd845d5bbc71958f8100581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:00.328833 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.329465 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.331049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1428/2848] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:00 2022] DELETE /identity/v3/projects/c4d2fd910dd845d5bbc71958f8100581 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:00.335046 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:00.344520 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcbe6-11dc-4bc4-b7f3-5a49253ed9c6 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:00.400434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:00.400973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:00.401248 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:00.401519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] PATH_INFO: `/v3/users/1a8e9aa35a164c1191613e8520529499` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:00.402631 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8e9aa35a164c1191613e8520529499)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:00.404550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:00.449251 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8e9aa35a164c1191613e8520529499'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.449896 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8e9aa35a164c1191613e8520529499'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.450496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8e9aa35a164c1191613e8520529499'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.457695 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8e9aa35a164c1191613e8520529499'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.469378 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-cd10b0ff-819e-4670-ba4b-dd2bf34ad2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8e9aa35a164c1191613e8520529499'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.473704 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:00.484612 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1421/2849] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:49:00 2022] DELETE /identity/v3/users/1a8e9aa35a164c1191613e8520529499 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:00.756597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:00.758108 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:00.772891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a49e-c76c-4048-910a-0a9b7914701d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:00.831138 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:00.831795 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:00.832042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:00.832320 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] PATH_INFO: `/v3/projects/ab40a6072b194ff0916249f8b70c9177` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:00.833453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab40a6072b194ff0916249f8b70c9177)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:00.835537 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:00.880251 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab40a6072b194ff0916249f8b70c9177', 'request_id': 'req-58e3ffba-e862-4c14-944a-59e284855620'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.886798 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab40a6072b194ff0916249f8b70c9177', 'request_id': 'req-58e3ffba-e862-4c14-944a-59e284855620'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.892818 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab40a6072b194ff0916249f8b70c9177', 'request_id': 'req-58e3ffba-e862-4c14-944a-59e284855620'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.905989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] The token cache is being invalidate because project ab40a6072b194ff0916249f8b70c9177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:00.906281 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.906936 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58e3ffba-e862-4c14-944a-59e284855620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:00.908418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1429/2850] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:00 2022] DELETE /identity/v3/projects/ab40a6072b194ff0916249f8b70c9177 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:00.913431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:00.913907 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:00.914011 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:00.933966 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:00.935142 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:00.993744 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1422/2851] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:01.001634 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.001921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.002200 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.022592 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:01.023951 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:01.031940 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1430/2852] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:01.036174 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.046026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d0f35-08c5-412c-815b-4b69bc6c2539 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.050556 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.051201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.051491 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.051766 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.052412 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:49:01.053257 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.054948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.066557 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1423/2853] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:49:01.071647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.080946 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0e470-645b-45b3-9e82-8b134890b598 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.085720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.086311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.086587 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.086891 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.087529 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.089589 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.118325 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1431/2854] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:49:01.122948 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.133598 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bff9-e9b5-46df-aa06-1dfbb138c90d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.190146 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.190683 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.190972 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.191249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.191880 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.194731 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.215679 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1424/2855] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:01.220109 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.230752 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4575-3bbe-4a2b-9fe1-bba014a92848 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.235337 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.235925 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.236204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.236475 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.237111 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.238720 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.248473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1432/2856] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:01.252552 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.274406 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff655-1dac-4b6f-a27c-de2bf901a142 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.279240 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.279838 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.280116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.280388 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.281028 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.282613 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc5059-6374-404d-a43c-b421bc04871d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.293756 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1425/2857] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:01.297958 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.307929 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da168-c869-4109-b2e5-a7ded3440a6d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.313042 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.313699 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.313980 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.314256 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] PATH_INFO: `/v3/projects/05dfd04f1ee14367a366c716335f83de/users/542da337a9b24fb89b173d7bfa9afbe2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.323483 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:01.330785 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dfd04f1ee14367a366c716335f83de, user_id=542da337a9b24fb89b173d7bfa9afbe2, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.333921 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.347044 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1433/2858] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:01 2022] PUT /identity/v3/projects/05dfd04f1ee14367a366c716335f83de/users/542da337a9b24fb89b173d7bfa9afbe2/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:01.359394 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.359686 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.359967 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.389620 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] MFA Rules not processed for user `542da337a9b24fb89b173d7bfa9afbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:01.390866 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:01.436212 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:01.455061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1426/2859] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:01.460258 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.520428 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac2e3-f8fb-4ab6-8800-f6e6f2c896fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.525776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.526375 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.526701 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.526979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.527615 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.529655 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.556089 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1434/2860] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:01.560321 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.571730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c748c1-5e9e-4ad7-a9c4-d4079a827cb4 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.626618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.627242 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.627521 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.627797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.628422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.630385 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.650571 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1427/2861] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:01.654773 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.665788 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0002-f131-4db9-8586-50004622c4af admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.670431 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.671045 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.671326 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.671598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.672236 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.673815 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.684651 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1435/2862] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:01.688931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.700821 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9a0d-b1b5-4725-a840-ab742456506a admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.705976 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.706566 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.706882 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.707159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] PATH_INFO: `/v3/projects/ab1ffc2340444df48b3246392c9e5c08/users/b90a69320f08413bb55057654c7c2021/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.722023 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1ffc2340444df48b3246392c9e5c08, user_id=b90a69320f08413bb55057654c7c2021, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.724709 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.737632 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1428/2863] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:01 2022] PUT /identity/v3/projects/ab1ffc2340444df48b3246392c9e5c08/users/b90a69320f08413bb55057654c7c2021/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:01.742510 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.752435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee153290-94a1-45ff-8705-1325a92d0103 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.807642 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.808247 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.808530 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.808805 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.809445 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.811081 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c5c3a-62fd-4871-a13c-e4a85b7c15bc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.821542 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1436/2864] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:49:01.825730 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.836167 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b1d95-0590-4cf6-88e0-360b57099f0d admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:01.840784 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:01.841445 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.841707 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.841980 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] PATH_INFO: `/v3/projects/ab1ffc2340444df48b3246392c9e5c08/users/b90a69320f08413bb55057654c7c2021/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.843894 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1ffc2340444df48b3246392c9e5c08, user_id=b90a69320f08413bb55057654c7c2021, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:01.846516 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:01.857140 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1429/2865] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:01 2022] PUT /identity/v3/projects/ab1ffc2340444df48b3246392c9e5c08/users/b90a69320f08413bb55057654c7c2021/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:01.869180 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:01.869703 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:01.869870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:01.897603 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] MFA Rules not processed for user `b90a69320f08413bb55057654c7c2021`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:01.898844 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:01.943040 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:01.969464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1437/2866] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:01.980973 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:01.982853 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:02.042040 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636483e-2e09-410d-988e-aae458d542ec admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:02.045661 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:02.046259 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:02.046538 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:02.046849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:02.099395 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:02.102263 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:02.104268 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:02.113381 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1430/2867] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:06.302364 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:06.303778 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:06.316234 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cce50-6542-40e3-807b-db35bf43788e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:06.319871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:06.320473 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:06.320772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:06.321025 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:06.330968 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:06.332394 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae1feb-a65a-4ab5-acb9-49366b667781 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:06.341895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1438/2868] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:11.214171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:11.275434 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072fafe-3dbf-4933-9bf9-d52438f9449d service cinder] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:11.280001 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:11.280649 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:11.280927 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:11.281201 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] PATH_INFO: `/v3/users/542da337a9b24fb89b173d7bfa9afbe2` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:11.282313 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542da337a9b24fb89b173d7bfa9afbe2)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:11.284261 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:11.341763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542da337a9b24fb89b173d7bfa9afbe2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.342422 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542da337a9b24fb89b173d7bfa9afbe2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.343052 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542da337a9b24fb89b173d7bfa9afbe2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.350817 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542da337a9b24fb89b173d7bfa9afbe2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.363840 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542da337a9b24fb89b173d7bfa9afbe2'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.376061 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1431/2869] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:49:11 2022] DELETE /identity/v3/users/542da337a9b24fb89b173d7bfa9afbe2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:11.380551 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:11.381037 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:11.381151 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:11.400712 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:11.401898 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:11.459671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1439/2870] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:11.470053 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:11.471532 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:11.529070 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0031d-7860-4a2f-9530-6ec0c8670738 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:11.532768 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:11.533372 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:11.533654 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:11.533931 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:11.543544 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:11.545004 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:11.554393 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1432/2871] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:11.820965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:11.823193 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:11.837998 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764ad4-3011-4c19-a66c-83cb62002250 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:11.842628 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:11.843279 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:11.843562 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:11.843835 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] PATH_INFO: `/v3/projects/05dfd04f1ee14367a366c716335f83de` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:11.845629 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05dfd04f1ee14367a366c716335f83de)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:11.847770 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:11.894885 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05dfd04f1ee14367a366c716335f83de', 'request_id': 'req-dab35a06-4487-429e-9736-2358277ebfd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.900939 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05dfd04f1ee14367a366c716335f83de', 'request_id': 'req-dab35a06-4487-429e-9736-2358277ebfd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.906938 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05dfd04f1ee14367a366c716335f83de', 'request_id': 'req-dab35a06-4487-429e-9736-2358277ebfd5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.918810 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] The token cache is being invalidate because project 05dfd04f1ee14367a366c716335f83de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:11.919097 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.919714 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:11.921199 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1440/2872] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:11 2022] DELETE /identity/v3/projects/05dfd04f1ee14367a366c716335f83de => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:11.925196 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:11.935416 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ce65d-e040-42ab-9fb6-2222fdaf0e14 service neutron] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:11.947668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:11.994401 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:11.995021 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:11.995303 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:11.995579 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] PATH_INFO: `/v3/users/b90a69320f08413bb55057654c7c2021` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:11.996693 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90a69320f08413bb55057654c7c2021)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:11.998607 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.044034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90a69320f08413bb55057654c7c2021'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.044681 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90a69320f08413bb55057654c7c2021'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.045283 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90a69320f08413bb55057654c7c2021'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.052539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90a69320f08413bb55057654c7c2021'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.065195 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-b159f4f2-eb94-415d-9613-8bf2afa4b327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90a69320f08413bb55057654c7c2021'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.077370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1433/2873] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:49:11 2022] DELETE /identity/v3/users/b90a69320f08413bb55057654c7c2021 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:12.346669 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.348144 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:12.362158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab35a06-4487-429e-9736-2358277ebfd5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.416524 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:12.427753 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.428454 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.428737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.429018 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] PATH_INFO: `/v3/projects/ab1ffc2340444df48b3246392c9e5c08` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.430169 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1ffc2340444df48b3246392c9e5c08)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.438861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.488590 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1ffc2340444df48b3246392c9e5c08', 'request_id': 'req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.494253 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1ffc2340444df48b3246392c9e5c08', 'request_id': 'req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.500435 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1ffc2340444df48b3246392c9e5c08', 'request_id': 'req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.513063 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] The token cache is being invalidate because project ab1ffc2340444df48b3246392c9e5c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:12.513349 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.513989 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3d2949ae-0065-4f49-9ba8-82d7470ec3eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:12.515585 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1441/2874] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:12 2022] DELETE /identity/v3/projects/ab1ffc2340444df48b3246392c9e5c08 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:12.521507 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.521982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.521982 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.541820 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:12.543034 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:12.600431 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1434/2875] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:12.608991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.609311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.609594 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.630586 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:12.631802 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:12.640501 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1442/2876] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:12.644891 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.655234 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6215-cd6d-4d2d-9f76-803c3625e30c None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.659763 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.660357 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.660636 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.660906 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.661547 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81446) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:49:12.661849 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.663462 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.674711 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1435/2877] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 15:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:49:12.679168 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.688598 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e290f-e8c7-4bd6-98c1-7c9f7e740429 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.693400 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.693991 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.694271 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.694543 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.695207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.697737 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.724624 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1443/2878] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:49:12 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:12.729125 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.738986 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae43ef2-6404-4abc-9082-0e8ae86056d8 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.760947 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:12.797457 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.797969 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.798247 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.798519 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.799172 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.801077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.822391 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1436/2879] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:49:12 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:12.826747 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.837280 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f67c-4d8d-42f0-b1a0-2463da42e282 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.841829 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.842421 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.842736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.843019 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.843660 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.845245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.855743 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1444/2880] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:12.859805 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.870305 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5fb2-c2b6-4cd9-96bd-507e073b652c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.874928 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.875517 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.875797 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.876072 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.876738 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.878314 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b7c88-39c3-4663-99e2-2e83b78d6633 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.889213 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1437/2881] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:12.893229 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:12.903145 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea00cea-a2d9-4e61-9753-796a03d9262d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:12.907709 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:12.908342 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.908627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.908900 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] PATH_INFO: `/v3/projects/25e16f3f8f894e59a584a42fc28063c1/users/b1d0e8fdabc64064b73304852ba51465/roles/fdb2cb2ca51e4f0eaa2683395176a5bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.923023 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e16f3f8f894e59a584a42fc28063c1, user_id=b1d0e8fdabc64064b73304852ba51465, role_id=fdb2cb2ca51e4f0eaa2683395176a5bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:12.925736 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:12.939207 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1445/2882] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:12 2022] PUT /identity/v3/projects/25e16f3f8f894e59a584a42fc28063c1/users/b1d0e8fdabc64064b73304852ba51465/roles/fdb2cb2ca51e4f0eaa2683395176a5bc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:12.952361 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:12.952646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:12.952924 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:12.981223 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] MFA Rules not processed for user `b1d0e8fdabc64064b73304852ba51465`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:12.982435 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:13.046500 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1438/2883] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:13.051117 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.112861 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49a013-08de-4f3e-9856-e4bcda236d30 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.117517 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.118724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.119013 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.119291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.119913 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.121819 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.147995 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1446/2884] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 15:49:13 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:13.152376 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.162862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d6bef-5e62-4bec-92d6-02fbad859ea3 None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.218383 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.219018 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.219323 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.219599 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.220217 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.222126 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.244185 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1439/2885] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 15:49:13 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:49:13.248478 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.255263 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:13.259635 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dcab4-81c8-421e-a670-34bb23abe58b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.265676 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.266312 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.266792 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.267124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.267961 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.269546 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.279960 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1447/2886] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:13.284049 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.294302 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bca71-d1a6-4181-9483-2fc0cef43ed4 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.298883 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.299524 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.299803 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.300076 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] PATH_INFO: `/v3/projects/d290c8fbe242468ba77656724b56d6c6/users/06354e5fb4d64cbdb941a8f393e2b7cd/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.315243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d290c8fbe242468ba77656724b56d6c6, user_id=06354e5fb4d64cbdb941a8f393e2b7cd, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.317822 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.326496 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:13.333102 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1440/2887] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:13 2022] PUT /identity/v3/projects/d290c8fbe242468ba77656724b56d6c6/users/06354e5fb4d64cbdb941a8f393e2b7cd/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:13.337087 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.347158 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8b51-9594-4016-bce8-1ca1d9e2b8ee admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.411115 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.411724 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.412007 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.412292 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.412965 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.414596 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f615c7f-4aaa-462c-9496-3b297eeeffd1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.424979 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1448/2888] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:49:13.429266 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.440241 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65592-a84a-4dbf-85c9-21e9f4acfed3 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.444965 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.445608 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] REQUEST_METHOD: `PUT` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.446005 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.446347 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] PATH_INFO: `/v3/projects/d290c8fbe242468ba77656724b56d6c6/users/06354e5fb4d64cbdb941a8f393e2b7cd/roles/826e2155943c48ab9c01f294b1734d83` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.448272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d290c8fbe242468ba77656724b56d6c6, user_id=06354e5fb4d64cbdb941a8f393e2b7cd, role_id=826e2155943c48ab9c01f294b1734d83)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.450952 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.461689 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1441/2889] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 15:49:13 2022] PUT /identity/v3/projects/d290c8fbe242468ba77656724b56d6c6/users/06354e5fb4d64cbdb941a8f393e2b7cd/roles/826e2155943c48ab9c01f294b1734d83 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:13.473871 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.475094 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.475357 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.503573 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] MFA Rules not processed for user `06354e5fb4d64cbdb941a8f393e2b7cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:13.504794 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:13.574245 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1449/2890] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:13.585309 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:13.587703 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:13.643643 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a486fd-5668-4179-9a50-3dbe021e814c admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:13.647287 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ebbe8c6f1734885ae15e7137ecf8495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:13.647878 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:13.648159 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:13.648432 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:13.708942 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:13.710365 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37de2c-9a79-4b4e-8b9c-822b47a7e5cb service cinder] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:13.719635 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1442/2891] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:16.247434 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:16.308550 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74746d8c-9fe3-42cc-b5b7-25771c6da4b5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:16.313214 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:16.313897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:16.314183 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:16.314464 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] PATH_INFO: `/v3/users/b1d0e8fdabc64064b73304852ba51465` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:16.315681 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d0e8fdabc64064b73304852ba51465)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:16.317627 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:16.334154 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:16.364380 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d0e8fdabc64064b73304852ba51465'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.365004 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d0e8fdabc64064b73304852ba51465'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.372576 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d0e8fdabc64064b73304852ba51465'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.373252 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d0e8fdabc64064b73304852ba51465'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.380776 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d0e8fdabc64064b73304852ba51465'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.397870 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1450/2892] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:49:16 2022] DELETE /identity/v3/users/b1d0e8fdabc64064b73304852ba51465 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:16.402618 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] REQUEST_METHOD: `POST` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:16.403135 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:16.403250 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:16.421915 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.auth.core [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] MFA Rules not processed for user `75821779c93e4039bcee682de1f82886`. Rule list: `[]` (Enabled: `True`). {{(pid=81446) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:49:16.423130 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:16.480708 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1443/2893] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:49:16.490369 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:16.491834 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:16.549448 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423234b2-ee39-460b-8872-f1c2a5872862 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:16.553150 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:16.553754 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:16.554035 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:16.554311 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:16.563300 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:16.564772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:16.574167 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1451/2894] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:49:16.851380 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:16.853464 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:16.867332 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099f4cb-dab5-47fd-afb5-ff7bd9a1ec3e None None] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:16.872921 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:16.873682 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:16.874026 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:16.874371 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] PATH_INFO: `/v3/projects/25e16f3f8f894e59a584a42fc28063c1` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:16.875540 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e16f3f8f894e59a584a42fc28063c1)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:16.877705 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:16.915341 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:16.926136 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e16f3f8f894e59a584a42fc28063c1', 'request_id': 'req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.932316 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e16f3f8f894e59a584a42fc28063c1', 'request_id': 'req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.938405 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e16f3f8f894e59a584a42fc28063c1', 'request_id': 'req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.950260 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] The token cache is being invalidate because project 25e16f3f8f894e59a584a42fc28063c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:16.950543 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.951200 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:16.952653 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1444/2895] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:16 2022] DELETE /identity/v3/projects/25e16f3f8f894e59a584a42fc28063c1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:16.956652 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:16.966474 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f4e8-0055-42f5-8312-9486df2238d3 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:17.022892 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:17.023446 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:17.023731 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:17.024009 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] PATH_INFO: `/v3/users/06354e5fb4d64cbdb941a8f393e2b7cd` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:17.025240 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06354e5fb4d64cbdb941a8f393e2b7cd)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:17.027223 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:17.070388 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:17.078197 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06354e5fb4d64cbdb941a8f393e2b7cd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.078897 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06354e5fb4d64cbdb941a8f393e2b7cd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.085661 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06354e5fb4d64cbdb941a8f393e2b7cd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.086290 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06354e5fb4d64cbdb941a8f393e2b7cd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.094909 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06354e5fb4d64cbdb941a8f393e2b7cd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.113645 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1452/2896] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:49:16 2022] DELETE /identity/v3/users/06354e5fb4d64cbdb941a8f393e2b7cd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:17.394498 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:17.396116 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:17.420929 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca81f8-df4d-41fe-a8b6-eea7edf193d1 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:17.476985 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:17.478272 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:17.478550 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:17.479137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] PATH_INFO: `/v3/projects/d290c8fbe242468ba77656724b56d6c6` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:17.480246 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d290c8fbe242468ba77656724b56d6c6)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:17.482133 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:17.527080 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd290c8fbe242468ba77656724b56d6c6', 'request_id': 'req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.532637 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd290c8fbe242468ba77656724b56d6c6', 'request_id': 'req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.538856 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd290c8fbe242468ba77656724b56d6c6', 'request_id': 'req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.551407 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] The token cache is being invalidate because project d290c8fbe242468ba77656724b56d6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:49:17.551700 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.552330 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:49:17.553815 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1445/2897] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:49:17 2022] DELETE /identity/v3/projects/d290c8fbe242468ba77656724b56d6c6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:49:27.110033 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:27.548438 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:20.555092 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:20.556536 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:20.737593 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b183147-507d-4df4-a4e7-31e3d825170c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:20.784531 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e2f7189b07f4024b7744322c707c2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:20.785124 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:20.785398 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:20.785666 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:20.787418 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:20.847358 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:20.848772 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:20.859082 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1453/2898] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:26.427944 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:26.429514 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:26.516615 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac33f-9f81-4bf8-9642-a14f0a6c4cd0 admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:26.521581 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:26.522220 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] REQUEST_METHOD: `GET` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:26.522499 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:26.522808 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:26.524535 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:26.536862 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:26.539077 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:26.550137 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1446/2899] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:27.192204 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:27.193659 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:27.207244 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36431b3-361b-4139-8be0-b30640f06b6a service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:27.211738 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:27.212359 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:27.212640 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:27.212914 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] PATH_INFO: `/v3/users/81be4c83d3b04624ae11bcbf37a44309` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:27.214067 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81be4c83d3b04624ae11bcbf37a44309)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:27.216055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:27.262597 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81be4c83d3b04624ae11bcbf37a44309'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.263283 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81be4c83d3b04624ae11bcbf37a44309'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.270222 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81be4c83d3b04624ae11bcbf37a44309'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.270860 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81be4c83d3b04624ae11bcbf37a44309'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.274472 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:27.279686 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81be4c83d3b04624ae11bcbf37a44309'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.298266 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1454/2900] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:27 2022] DELETE /identity/v3/users/81be4c83d3b04624ae11bcbf37a44309 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:27.471639 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:27.483646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be04a2-584e-43db-b68f-70a4dd7f91b6 service placement] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:27.544389 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:27.545020 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:27.545298 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:27.545574 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] PATH_INFO: `/v3/projects/b496441965834f65a8d0813e8221c685` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:27.546733 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b496441965834f65a8d0813e8221c685)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:27.548638 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:27.593587 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b496441965834f65a8d0813e8221c685', 'request_id': 'req-57e83883-2a75-42db-88fa-4d8354ad738f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.598993 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b496441965834f65a8d0813e8221c685', 'request_id': 'req-57e83883-2a75-42db-88fa-4d8354ad738f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.605282 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b496441965834f65a8d0813e8221c685', 'request_id': 'req-57e83883-2a75-42db-88fa-4d8354ad738f'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.619106 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] The token cache is being invalidate because project b496441965834f65a8d0813e8221c685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:27.619611 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.620646 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.623243 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1447/2901] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:27 2022] DELETE /identity/v3/projects/b496441965834f65a8d0813e8221c685 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:50:27.628671 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:27.630174 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:27.643854 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52111174-e1e7-4c17-b1df-79bb26589a48 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:27.698685 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:27.699355 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:27.700075 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:27.700347 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] PATH_INFO: `/v3/users/c22dc9314f7648a3a7d25b12fc351a4f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:27.701481 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c22dc9314f7648a3a7d25b12fc351a4f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:27.703453 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:27.748386 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c22dc9314f7648a3a7d25b12fc351a4f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.749055 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c22dc9314f7648a3a7d25b12fc351a4f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.756310 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c22dc9314f7648a3a7d25b12fc351a4f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.756947 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c22dc9314f7648a3a7d25b12fc351a4f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.764631 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c22dc9314f7648a3a7d25b12fc351a4f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:27.781842 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1455/2902] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:27 2022] DELETE /identity/v3/users/c22dc9314f7648a3a7d25b12fc351a4f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:27.951171 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Authenticating user token {{(pid=81446) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:27.952548 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.fernet_utils [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81446) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:27.965723 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83883-2a75-42db-88fa-4d8354ad738f admin admin] Validating token access rules against request {{(pid=81446) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:27.984269 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:28.029539 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75821779c93e4039bcee682de1f82886', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e88980106c84e4fadc67759de8e9449', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81446) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:28.030177 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:28.030461 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:28.030794 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] PATH_INFO: `/v3/projects/e232defe64dc4838abc09dbe8d1df3b3` {{(pid=81446) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:28.031930 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e232defe64dc4838abc09dbe8d1df3b3)` {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:28.033848 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] RBAC: Authorization granted {{(pid=81446) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:28.087370 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e232defe64dc4838abc09dbe8d1df3b3', 'request_id': 'req-3cdac653-8b9c-4d41-ae13-862b433fc19e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:28.094668 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e232defe64dc4838abc09dbe8d1df3b3', 'request_id': 'req-3cdac653-8b9c-4d41-ae13-862b433fc19e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:28.102190 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e232defe64dc4838abc09dbe8d1df3b3', 'request_id': 'req-3cdac653-8b9c-4d41-ae13-862b433fc19e'} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:28.116338 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] The token cache is being invalidate because project e232defe64dc4838abc09dbe8d1df3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81446) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:28.116624 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:28.117249 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG keystone.notifications [None req-3cdac653-8b9c-4d41-ae13-862b433fc19e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81446) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:28.118813 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: [pid: 81446|app: 0|req: 1448/2903] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:27 2022] DELETE /identity/v3/projects/e232defe64dc4838abc09dbe8d1df3b3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:50:37.778540 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:38.112923 dv-cinder-846054-2-iscsi devstack@keystone.service[81446]: DEBUG dbcounter [-] [81446] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81446) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:55:30.324230 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:55:30.325662 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:55:30.471811 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674743-6d3e-4ba7-9196-8a3ccf8558d9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:55:30.534895 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5edde502e37144e1b36c7fcfadb6acdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e025a9fe2c2249c99087d3d22603d623', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:55:30.535848 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:55:30.536277 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:55:30.536553 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:55:30.539166 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:55:30.553647 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:55:30.555291 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3e26-eaa8-447b-a019-61fa8d4b3cbb service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:55:30.565049 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1456/2904] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 15:55:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:55:40.560837 dv-cinder-846054-2-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}